IPB

Welcome Guest ( Log In | Register )

4 Pages V  « < 2 3 4  
Reply to this topicStart new topic
Lossless Extensions for Opus (Backwards Compatible), How to embed lossless deltas inside an Opus stream
C.R.Helmrich
post Mar 29 2013, 15:10
Post #76





Group: Developer
Posts: 686
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (wswartzendruber @ Mar 29 2013, 04:37) *
I'm not going to stop until I see for myself that this will not work well.

This is an exemplary attitude! Sometimes I wish we had a bit more of this in the scientific and engineering world, since it's often the only way to achieve progress.

Honestly, good luck with your project. And please, do let us know about your findings.

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
wswartzendruber
post Mar 30 2013, 22:58
Post #77





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



I'll just use this thread as sort of a log.

Anyway, I've tested Monkey's Audio against Opus residuals at a variety of bitrates and it is proving to be competitive against OptimFROG. Given that I have no idea what lossless codec may eventually be used to compress the residuals, I need to redo the principles of operation to accommodate one that uses variable frame sizes (that is, other than 60 ms, which I have been assuming up until now). It is highly likely that there will be many Opus packets for each delta packet, and this ratio is likely to vary throughout a stream.
Go to the top of the page
+Quote Post
Dynamic
post Mar 31 2013, 10:54
Post #78





Group: Members
Posts: 803
Joined: 17-September 06
Member No.: 35307



With mention of an open lossy codec being a requirement, the other option is Ogg Vorbis, which supports specific sampling rates and isn't too far behind Opus in bitrate efficiency and is an open format that has fairly widespread support in the wild (main exception being Apple, despite Wikipedia's use of Vorbis). Then you have no worry about resampling consistently so that the hybrid remains lossless. You then just need to presumably place non-Vorbis packets in the Ogg stream that all Vorbis decoders (except your hybrid decoder) will ignore. Presumably it would also be easy to strip out only Vorbis packets from the Ogg stream to obtain a smaller lossy file.

The other option where sampling rate can be specified and still be expected to play is Xiph.org's CELT (forerunner to the CELT half of Opus you're thinking of), but plain CELT decoder support in the wild is very limited and will very likely remain so now Opus has superceded it, even though I believe Google's plugin for Chat/Hangouts uses CELT in its new Studio mode (click Learn More in this link) and will possibly implement Opus in due course, possibly piggybacking on WebRTC integration.

On the positive side for Opus, it's possible that being standardized by IETF with clearly specified open patent licensing, support will eventually surpass that of Vorbis.
Go to the top of the page
+Quote Post
Destroid
post Mar 31 2013, 11:55
Post #79





Group: Members
Posts: 548
Joined: 4-June 02
Member No.: 2220



Dynamic's suggestion is very interesting. In addition to the points mentioned there are a few other things I wanted rehearse here:

- former lossless Vorbis [q 10] discontinued;
- new iteration of Vorbis, possibly ditched due to Ghost/OPUS and the possibility of broken compatibility;
- the AoTuV project low-bitrate settings (q-1, q-2) sounded better to me than Speex at comparable bitrate, IMO.


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
wswartzendruber
post Mar 31 2013, 15:40
Post #80





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



One of the main reasons for choosing Opus is because of the IETF standardization Dynamic mentioned. Also, Opus lets me extend it at the sub-container level. There should be no reason a lossless/hybrid stream can't be placed in any container that can handle standardized Opus. My hybrid encoder API so far is completely container agnostic.

This post has been edited by wswartzendruber: Mar 31 2013, 15:40
Go to the top of the page
+Quote Post
wswartzendruber
post Apr 1 2013, 22:52
Post #81





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



Florin got back to me. He's just in a bit of a personal dilemma and estimates reviewing my comments in one week.

This post has been edited by wswartzendruber: Apr 1 2013, 22:52
Go to the top of the page
+Quote Post
wswartzendruber
post Apr 19 2013, 06:38
Post #82





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



One week, right. I think he's in a bit more of a jam than he admitted.

Whatever. As soon as libopus-1.1 comes out, I'm gonna start gutting the tree until I have a fixed-point decoder and nothing else.
Go to the top of the page
+Quote Post
wswartzendruber
post Aug 9 2013, 22:24
Post #83





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



QUOTE (wswartzendruber @ Apr 19 2013, 01:38) *
One week, right. I think he's in a bit more of a jam than he admitted.

Whatever. As soon as libopus-1.1 comes out, I'm gonna start gutting the tree until I have a fixed-point decoder and nothing else.

All right, now that I've graduated from college (Bachelor in Computer Science) and have myself a job, I can get back to the task at hand. After having thought about this for a while, I think the best answer may just be to implement my own Opus (CELT) decoder in fixed-point arithmetic. This is going to really suck, but I need stability with a base I understand. I also don't see myself needing the SILK parts of the reference implementation. I should know more this weekend after I've gone over the appropriate parts of the RFC.
Go to the top of the page
+Quote Post
wswartzendruber
post Nov 30 2013, 21:08
Post #84





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



Florin finally got back to me! He's been all tied up with real-life stuff. Moving and whatnot. He still seems determined to help. I won't share what he told me because I don't have his permission to do so, but I'll share what I said to him:

QUOTE
Hi Florin,

I'm on my tablet so I won't type a lot. I have outlined some main points below.

1. Thank you for getting back to me.

2. I am in the United States (California).

3. I have determined that we should have our own implementations of Opus.

4. We only need the MDCT-based CELT layer.

5. I don't know how to do any of this, so I am learning from the ground up. I have just finished, as practice, my first FFT implementation, and then used that to create (from scratch) an FFT-based convolver that uses overlap-and-add. In the coming days I will move onto learn and implement the MDCT transform.

6. I hate Jean-Marc for trying to shoot this down so fast, but he has a point. Extension packets need to be standardized first.

7. I've lost the work I did, but it wasn't applicable anyway. I've since started using Git.

8. I still have every intention of making this happen.

9. You can track my learning progress here: https://github.com/wswartzendruber/logic/

-William
Go to the top of the page
+Quote Post
darkbyte
post Aug 7 2014, 10:52
Post #85





Group: Members
Posts: 149
Joined: 14-June 11
Member No.: 91517



This stuff died quickly. rolleyes.gif

Nevertheless i wonder why CELT was not designed in mind to be extensible to lossless. Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable. I wonder why audio codecs are not developed with this criteria in mind. Would it reduce CELT's efficiency considerably?


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
wswartzendruber
post Aug 7 2014, 14:09
Post #86





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



QUOTE (darkbyte @ Aug 7 2014, 05:52) *
This stuff died quickly. rolleyes.gif

Nevertheless i wonder why CELT was not designed in mind to be extensible to lossless. Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable. I wonder why audio codecs are not developed with this criteria in mind. Would it reduce CELT's efficiency considerably?

This isn't dead. I just haven't come back to run my mouth because I don't have anything besides wrapper stuff to show for it, and upon further consideration, that code has to change. The wrapper code was written to put CELT and lossless data in different packets. That was before I understood how multistream worked. Now that idea just seems ridiculous. Also, there's no way I'm trusting stuff like mkvmerge to say, "Hey, it's a packet full of jibberish with no real Opus data in it. Yes, I will mux this in."

No, the lossless deltas will be embedded directly in the padding for each packet. The wrapper around libopus will be written as a proof of concept that I can embed junk in Opus packets and get away with it in a variety of scenarios. Then after that, I will start on my own CELT implementation and the fun stuff will begin...

There seems to be some sort of decoder state I will have to synthesize when seeking. That might be a nail. If I give up on this project, everyone here will know.

This post has been edited by wswartzendruber: Aug 7 2014, 14:14
Go to the top of the page
+Quote Post
darkbyte
post Aug 7 2014, 14:35
Post #87





Group: Members
Posts: 149
Joined: 14-June 11
Member No.: 91517



Okay sorry, it's just there was no update for months, but now i understand.

I don't know by the way which is the better path. Putting correction data into the same container, the same way MPEG4-SLS do by mp4, or into a separate file how WavPack Hybrid does it. The latter makes very easy to copy only the lossy part to somewhere else but the number of files explodes if you have lot of audio on your drive. The first requires a tool to copy only the lossy part.

This post has been edited by darkbyte: Aug 7 2014, 14:36


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
wswartzendruber
post Aug 7 2014, 19:13
Post #88





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



I've already decided on using the same container. Makes for simpler management. I had trouble deciding if I wanted the deltas in the same packets as the lossy CELT bits or if I wanted them in their own extension packets.

EDIT: JMV has expressed concern that I'm creating a rogue extension. Each padding area will start off with its own identifier like "wswartzendruber_whacked_out_experiment" or some such thing. If future, standards-compliant decoders that support some new official extension mistake my lossless delta bits for something else, that's on him and the IETF.

EDIT: Stuffing the lossless deltas in the padding of the CELT packets also absolves some time stamping issues I've been concerned about when it comes to Ogg encapsulation.

This post has been edited by wswartzendruber: Aug 7 2014, 19:32
Go to the top of the page
+Quote Post
saratoga
post Aug 7 2014, 19:28
Post #89





Group: Members
Posts: 4903
Joined: 2-September 02
Member No.: 3264



QUOTE (darkbyte @ Aug 7 2014, 05:52) *
This stuff died quickly. rolleyes.gif

Nevertheless i wonder why CELT was not designed in mind to be extensible to lossless. Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable.


I wouldn't say that video codecs are more complex. Since the input data size is is ~ 3 orders of magnitude larger, and the encoding speed isn't exponentially slower, the complexity is probably much lower than a typical audio codec. Theres just a lot more data to process.

QUOTE (darkbyte @ Aug 7 2014, 05:52) *
I wonder why audio codecs are not developed with this criteria in mind. Would it reduce CELT's efficiency considerably?


Lossless mode in H.264 basically skips most of the codec and just does lossless packing. Not quite the same as being extensible to lossless, more like just including two different codecs in one format. I think from the point of view of CELT, there is a lossless mode, and it is called "FLAC".
Go to the top of the page
+Quote Post
jmvalin
post Aug 7 2014, 19:34
Post #90


Xiph.org Speex developer


Group: Developer
Posts: 479
Joined: 21-August 02
Member No.: 3134



QUOTE (wswartzendruber @ Aug 7 2014, 14:13) *
EDIT: JMV has expressed concern that I'm creating a rogue extension. Each padding area will start off with its own identifier like "wswartzendruber_whacked_out_experiment" or some such thing. If future, standards-compliant decoders that support some new official extension mistake my lossless delta bits for something else, that's on him and the IETF.


So what you're saying is that the IETF is no longer free to have an extension system based on an 8-bit ID because it would make collisions likely (in your example, if an 8-bit ID were "w" there would be a collision). That pretty much fits the definition of a rogue extension.
Go to the top of the page
+Quote Post
wswartzendruber
post Aug 7 2014, 20:07
Post #91





Group: Members
Posts: 90
Joined: 11-December 06
Member No.: 38563



QUOTE (jmvalin @ Aug 7 2014, 14:34) *
So what you're saying is that the IETF is no longer free to have an extension system based on an 8-bit ID because it would make collisions likely (in your example, if an 8-bit ID were "w" there would be a collision). That pretty much fits the definition of a rogue extension.

What I'm saying is that you've been getting in my way from day one and that's about all you've been doing. I have approached you, very tactfully, in the Opus mailing list about how to best have an extension. I recall you telling me to get lost.

Despite what you may think, I really do care about where the IETF is going to take Opus and I do wish to remain compatible. That is why I approached you. But if your only response is "get over it" and you guys somehow decide that an 8-bit extension ID without a null terminator is a good idea, then yes, it will be a rogue extension.

Tell me something... After openly standardizing a royalty-free codec, how did you not realize this would eventually happen?

This post has been edited by wswartzendruber: Aug 7 2014, 20:11
Go to the top of the page
+Quote Post
saratoga
post Aug 7 2014, 20:12
Post #92





Group: Members
Posts: 4903
Joined: 2-September 02
Member No.: 3264



I think people generally expect poorly thought out extensions that break things. There is nothing you can do to completely prevent them and historically a great many have cropped up for popular formats.

The best you can do is discourage them.
Go to the top of the page
+Quote Post
jmvalin
post Aug 11 2014, 19:10
Post #93


Xiph.org Speex developer


Group: Developer
Posts: 479
Joined: 21-August 02
Member No.: 3134



QUOTE (wswartzendruber @ Aug 7 2014, 15:07) *
Tell me something... After openly standardizing a royalty-free codec, how did you not realize this would eventually happen?


When winter arrives, I realize I'm likely to catch a cold. Doesn't mean I look forward to it.

On a different topic, how about you start by showing us how great you new extension works in practice and then I may take you a little bit more seriously.
Go to the top of the page
+Quote Post
skamp
post Aug 11 2014, 20:13
Post #94





Group: Developer
Posts: 1430
Joined: 4-May 04
From: France
Member No.: 13875



QUOTE (jmvalin @ Aug 11 2014, 20:10) *
QUOTE (wswartzendruber @ Aug 7 2014, 15:07) *
how did you not realize this would eventually happen?


When winter arrives, I realize I'm likely to catch a cold. Doesn't mean I look forward to it.




--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
2Bdecided
post Aug 12 2014, 13:41
Post #95


ReplayGain developer


Group: Developer
Posts: 5089
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (darkbyte @ Aug 7 2014, 10:52) *
Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable.
The adoption and use of scalable coding is almost non-existent. If you count MVC (using just the left image to get 2D, and both to get 3D) then that's one (tenuous) example.

As for scalable to lossless, does anyone have a single example of where a significant number of people actually use this?

I know there are standards for scalable, and scalable to lossless - and I know people play with this stuff - and I know of some situations where it even kind of makes sense (e.g. wavepack lossy) - but even then it's a niche of a niche. Doing it from a psychoacoustic/psycho-visual lossy core makes very little sense and becomes a niche of a niche of a niche.

Don't get me wrong - it's great fun to play with this, and you'll always learn something by experimenting - but please understand there's at least one very good reason why almost no one is using this approach. If you want to solve the problem of lossless/lossy co-existance, there are other things you could do that more people would use - you might even be the first to do them exceptionally well based on Opus. e.g.
http://sysadminsjourney.com/content/2008/1...-mp3-fly-mp3fs/
https://github.com/robinbowes/flac2mp3
or some new idea.

Cheers,
David.
Go to the top of the page
+Quote Post
skamp
post Aug 12 2014, 14:03
Post #96





Group: Developer
Posts: 1430
Joined: 4-May 04
From: France
Member No.: 13875



QUOTE (2Bdecided @ Aug 12 2014, 14:41) *
If you want to solve the problem of lossless/lossy co-existance, there are other things you could do that more people would use - you might even be the first to do them exceptionally well based on Opus. e.g.
http://sysadminsjourney.com/content/2008/1...-mp3-fly-mp3fs/
https://github.com/robinbowes/flac2mp3
or some new idea.


I don't understand what transcoding software has to do with "scaling"?

Never mind, I misread your post.

This post has been edited by skamp: Aug 12 2014, 14:10


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post

4 Pages V  « < 2 3 4
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st August 2014 - 07:53