IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Reply to this topicStart new topic
lossless mp3 - mp3HD, backwards compatible with standard mp3
2Bdecided
post Mar 19 2009, 17:33
Post #1


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE
  • mp3HD is a lossless audio codec (100% bit-exact replica of CD tracks)
  • Backward Compatible to mp3
  • File extension .mp3
  • Bitrates for music approximately 500 to 900 kbps rates (similar to other lossless codecs), depending on genre
  • Embedded mp3 track and the mp3HD file share the same id3 metadata
  • Encoding parameters (e.g. bit rate), ancillary data and meta data of embedded mp3 track are under control


Main page:
http://www.all4mp3.com/Learn_mp3_hd_1.aspx

...which links to downloadable test software:
http://www.all4mp3.com/Software3.aspx

I learnt about this from this post...
http://forum.doom9.org/showthread.php?p=1263465#post1263465
...where a quick test suggests it's not as efficient as FLAC, or as fast.

Might be useful for a widely-compatible lossy/lossless hybrid.

I wonder if they've done something clever - when I tried this years ago, mp3 was a terrible base for lossless coding - the lossy>lossless "correction" file was often larger than a straight lossless encode. Maybe this new technology is more tightly integrated into mp3, so performs better?

Cheers,
David.

This post has been edited by 2Bdecided: Mar 19 2009, 17:36
Go to the top of the page
+Quote Post
ameyer17
post Mar 19 2009, 21:17
Post #2





Group: Banned
Posts: 78
Joined: 7-October 04
Member No.: 17551



QUOTE (2Bdecided @ Mar 19 2009, 09:33) *
I wonder if they've done something clever - when I tried this years ago, mp3 was a terrible base for lossless coding - the lossy>lossless "correction" file was often larger than a straight lossless encode. Maybe this new technology is more tightly integrated into mp3, so performs better?

Cheers,
David.

If I had to guess, I'd say that the mp3 encoder in question is designed to minimize the mathematical differences with the original file, as opposed to various modern mp3 codecs that are designed to minimize the acoustic differences.

And I question the backward compatibility. I encoded a 3 minute 11 second song and mplayer is reporting that it's 46:28 long.
And the quality of the "VBR highest quality" mode isn't particularly good. (ABX test to follow, I hope)
The efficiency isn't that good, either. That 3:11 song is approximately 1.3 MB bigger, too.

This post has been edited by greynol: Apr 2 2009, 19:41
Go to the top of the page
+Quote Post
saivert
post Mar 19 2009, 21:27
Post #3





Group: Members
Posts: 167
Joined: 13-June 06
Member No.: 31801



This format must for the love of everything good not succeed. Please let it die. Do not even talk about it.
There is not reason to have a hybrid lossless/lossy format. And if you want that you already have WavPack's Hybrid Lossy mode.


--------------------
http://foobar2000.saivert.com - foobar2000 component repository
Go to the top of the page
+Quote Post
LANjackal
post Mar 19 2009, 23:23
Post #4





Group: Members
Posts: 731
Joined: 26-October 05
From: Various networks
Member No.: 25371



If it were really backward compatible, why does it need a plugin for playback? And why is it restricted to Windows. I'm gonna have to call BS on this one ...


--------------------
EAC>1)fb2k>LAME3.99 -V 0 --vbr-new>WMP12 2)MAC-Extra High
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 00:02
Post #5





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (LANjackal @ Mar 19 2009, 23:23) *
If it were really backward compatible, why does it need a plugin for playback? And why is it restricted to Windows. I'm gonna have to call BS on this one ...

A specific decoder is certainly needed to decode the lossless stream ; but any standard decoder would be able to decode the lossy part - hence the need for a specific decoder to get the best quality.
It sounds interesting - at least if performance (ratio) is on par with other formats. It should at least seduce people dealing with two different libraries (lossless for home - lossy for car or DAP).

Was someone able to make it work with foobar2000's converter? I could only get a regular mp3 file (128 kbps according to filesize) /-(

This post has been edited by guruboolez: Mar 20 2009, 00:22
Go to the top of the page
+Quote Post
Silversight
post Mar 20 2009, 00:31
Post #6





Group: Members
Posts: 310
Joined: 5-April 06
From: Aachen, Germany
Member No.: 29203



QUOTE (guruboolez @ Mar 20 2009, 00:02) *
Was someone able to make it work with foobar2000's converter? I could only get a regular mp3 file (128 kbps according to filesize) /-(

Strangely enough, it works with foobar2000's Converter if you force it to use another file extension. I told fb2k to give it an mp5 extension and the result was a fully functional MP3HD file which, after being renamed to *.mp3, could be played in fb2k (the normal MP3 part, that is) and decoded to the original Wave file by the MP3HD decoder.


--------------------
Nothing is impossible if you don't need to do it yourself.
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 00:56
Post #7





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



@Silversight
Good point! It works. I guess that tagging process is the cause of the problem. Once the "mp5" file is created and renamed to mp3 (at this moment, the file has no tags), and after you add some tags, the file is shrinked by foobar2000 and all the lossless part is gone (but the mp3 stream is still valid).
I'm not technician, but I guess that MP3HD file are built like: MP3 then CORRECTION or MP3 then LOSSLESS.

This post has been edited by guruboolez: Mar 20 2009, 00:57
Go to the top of the page
+Quote Post
ameyer17
post Mar 20 2009, 02:36
Post #8





Group: Banned
Posts: 78
Joined: 7-October 04
Member No.: 17551



QUOTE (ameyer17 @ Mar 19 2009, 13:17) *
If I had to guess, I'd say that the mp3 encoder in question is designed to minimize the mathematical differences with the original file, as opposed to various modern mp3 codecs that are designed to minimize the acoustic differences.

For what it's worth, there appears to be a 17 kHz lowpass, so perhaps not.

QUOTE (guruboolez @ Mar 19 2009, 16:02) *
It sounds interesting - at least if performance (ratio) is on par with other formats. It should at least seduce people dealing with two different libraries (lossless for home - lossy for car or DAP).


Yeah, the best of both worlds, bigger than flac and lossy (if not dealing with the mp3hd decoder)
The one song I encoded was approximately 800 kbps, whereas the flac -8 file is approximately 740 kbps.
Encoding speed seems to be about half that of flac.
Decoding speed is about 28x, whereas flac decodes at about 192x or so.

(All comparisons with flac use the 32 bit Linux version of mp3hd as downloaded from all4mp3.com and flac as packaged in the 64 bit version of Ubuntu 8.10 on a computer with an Intel E7400 processor and 4 GB of RAM)

This post has been edited by greynol: Apr 2 2009, 19:02
Go to the top of the page
+Quote Post
Destroid
post Mar 20 2009, 02:49
Post #9





Group: Members
Posts: 545
Joined: 4-June 02
Member No.: 2220



Awhile back I wondered who would be the first to implement MP3 with correction, and now it has happened.

It works as expected and all the media players have no problem playing the resulting MP3HD files. Original files are identical to MP3HD decodes. The MP3HD compresses around the level of FLAC -3, although much slower (as expected).

Interestingly, in my limited tests, the resulting filesize seems to go up with the lossy bitrate.


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 13:01
Post #10





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



I investigate a bit further with this encoder.
• First, tag edition is currently ruining the lossless (or correction) datas. That's why foobar2000 doesn't work without forcing another extension (like mp5): it adds tags at the end of the conversion and therefore deletes the all "HD" informations. I created several MP3HD files, and changed tags with fb2k, dbpoweramp and mp3tag: each time files were shrinked by ~80% ohmy.gif
With previous versions of foobar2000, user can disable tag writing after conversion: the encoder is therefore functional even with "mp3" extension.

• Then, I tried to see if MP3HD append an independant lossless file to the MP3 stream or if it really corresponds to a correction stream (like with WavPack and OptimFrog). For that purpose, I encoded the same file with different lossy targets (128 kbps, 320 kbps and also the highest VBR mode). Measuring the size of the "HD" data is easy once you can easily get the "lossy" part (with a simple tag edition as mentioned before). The size of the "HD" data changes according to bitrate. As a conclusion, it seems logical that the HD part is really a correction part, not an independant lossless stream append the a common MP3 file. The higher the lossy bitrate is, the lower the correction part is. In other words, MP3HD would bring some efficiency over a dual library (MP3 + FLAC for example).

My tests were based on two radically different files. The first one is a big hard rock track (several tracks merged) with an important lossless bitrate (>1100 kbps); the second is a long stereo John Cage composition including a lot of silence part and therefore encoded at an amazing 194 kbps bitrate with flac.

Click to enlarge the table:

or direct link : http://img5.imageshack.us/img5/9286/mp3hd.png

As a conclusion, I would say that MP3HD brings interesting but not amazing performance for people dealing with an independant MP3library. But to be really interesting it needs a simple software tool able to transfer to any portable player the lossy part of the lossless file. Unfortunately, I wouldn't recommand anyone using this encoder at the moment as your lossless collection could too easily be lost with a basic masstagging operation.

This post has been edited by guruboolez: Mar 20 2009, 13:51
Go to the top of the page
+Quote Post
menno
post Mar 20 2009, 15:00
Post #11


Nero MPEG4 developer


Group: Developer (Donating)
Posts: 1218
Joined: 11-October 01
From: LA
Member No.: 267



If you want to know how it works, look at MPEG-4 SLS, this is the same, but with MP3 as core codec. Indeed it will only give extra efficiency if you plan to have a library of both lossy and lossless versions.
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 15:54
Post #12





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Thanks menno.
Before someone asks, MP3HD doesn't benefit from lossywav (resulting file was even bigger on my test).

I also noticed two different issues:
- conversion failed with long tracks (> 1 hour). I used fb2k as frontend.
- with long tracks and therefore huge amount of correction file, it could take some time before some program could access to tags (noticed it with foobar2000 and DbPowerAmp tag editor). I read in a french site that correction data are written as IDTag. Is it possible (ID3 size limitation maybe)?
Go to the top of the page
+Quote Post
menno
post Mar 20 2009, 16:11
Post #13


Nero MPEG4 developer


Group: Developer (Donating)
Posts: 1218
Joined: 11-October 01
From: LA
Member No.: 267



QUOTE (guruboolez @ Mar 20 2009, 06:54) *
Thanks menno.
Before someone asks, MP3HD doesn't benefit from lossywav (resulting file was even bigger on my test).

I also noticed two different issues:
- conversion failed with long tracks (> 1 hour). I used fb2k as frontend.
- with long tracks and therefore huge amount of correction file, it could take some time before some program could access to tags (noticed it with foobar2000 and DbPowerAmp tag editor). I read in a french site that correction data are written as IDTag. Is it possible (ID3 size limitation maybe)?


Yep, the correction data is in the ID3 tag, because mp3 doesn't allow framesizes big enough for this. Maybe freeformat would work, but I guess hardly any players would play the mp3 file that way. And it would be very difficult to extract the original MP3 data. In MPEG-4 SLS, the lossless data is simply stored in a separate track in the MP4 file.
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 16:50
Post #14





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Thank you again menno for this clarification smile.gif
I wonder how difficult it will be to make MP3 tagging programs fully compatible with MP3HD. And how many people will lose the correction data by using a wrong tagging program (including audio players adding stats or stuff like that).

In my opinion, MP3HD was really useful and maybe designed for popular stores (like amazon) to give them the opportunity to sell lossless music accessible to everyone (as many customers don't even know file extension, so program installation or file conversion is not an option for biggest stores). But the bad interaction with the current software environment (tagging program and into some extend media players) is IMO ruining a bit this idea.
Go to the top of the page
+Quote Post
callisto
post Mar 20 2009, 17:21
Post #15





Group: Members
Posts: 362
Joined: 2-September 06
From: Berlin, Germany
Member No.: 34723



When many customers doesn't even know file extension why would they care if they have a lossless mp3hd? they just see .mp3 and wonder why it's so big. and that lossless part is just accesible with appropriate decoder, which has to be installed.
and I don't understand why it's mp3hd that gives stores like amazon that opportunity? they could also just use flac...

for me, reading this thread tells me that mp3hd is just useless. mp3 an already old and inferior compression format becoming lossless... oh well >_>


--------------------
fb2k on OSX: flac q8 > rockboxed Sansa e280v1: Vorbis q5.0
Go to the top of the page
+Quote Post
Peter
post Mar 20 2009, 17:34
Post #16


foobar2000 developer


Group: Admin
Posts: 3275
Joined: 30-September 01
Member No.: 84



Useless format:
  • The lossless part is stored in ID3v2 tags.
  • Size of ID3v2 tags is limited to 256MB by specifications; as a result, lossless part of an mp3hd file can't be larger than 256MB.
I wonder what those people will think of next. Maybe resurrect VQF or something.

Addendum:

Current tagging software isn't prepared to deal with this kind of situation, so you're going to see various disturbing behaviors such as:
  • Very slow tag updates (near-full-file-rewrite with each edit).
  • Heavy memory usage of tag editors.
  • Retagging stripping correction data.
  • Tag editing or even reading failures when approaching the 256MB limit because software will try to put each ID3v2 frame in a single memory block and allocating a single block of such size is likely to fail in 32-bit address space because of fragmentation issues.


This post has been edited by Peter: Mar 21 2009, 16:38
Reason for edit: Expanded and remastered.
Go to the top of the page
+Quote Post
rpp3po
post Mar 20 2009, 17:47
Post #17





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



QUOTE (Peter @ Mar 20 2009, 17:34) *
Useless format


Totally agree. Not having separate correction files is pretty bad, anyway. With mp3hd you can fit just one album on a CDR for your car stereo in the worst case. That's not the point of mp3.

I would propose a generic correction file creator for all lossy formats ("DeltaFlac"? rolleyes.gif). A very simple piece of software or plugin could decode a lossy stream (including gap correction) and generate a delta stream against the original PCM, which would then be FLAC compressed and stored somewhere safe (for the reconversion of a whole collection in 2025 to Super-AAC2-Pro-3000).

A realtime playback filter, e. g., for Foobar could just reverse the chain. Delta and lossy file could be matched by a simple tag for the case that filenames get messed up.

This post has been edited by rpp3po: Mar 20 2009, 17:56
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 17:59
Post #18





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (callisto @ Mar 20 2009, 17:21) *
When many customers doesn't even know file extension why would they care if they have a lossless mp3hd?

Having few knowledge about computer things doesn't mean that people don't care about quality and would buy anything. If advertising on "CD-Quality" is now pointless and confusing (because stores were claiming it since the beginning) selling "HD MP3" and advertising on files "four time bigger" (hence better) makes things easy to understand even to the less-technician people on earth.

QUOTE
and that lossless part is just accesible with appropriate decoder, which has to be installed.

It doesn't matter people could really access to the correction stuff or not. Most people (even here) wouldn't notice the difference between a truncated "HD" stream at 256 kbps and the full lossless music. I'm pretty sure placebo effect will ensure people that their HD file are now much better than regular MP3, even if the stream is the same.

QUOTE
and I don't understand why it's mp3hd that gives stores like amazon that opportunity? they could also just use flac...

Yes, and having their hotline full of people unable to make their flac/alac/WMAlossless work on their computer, digital players or even both. Selling a format customers don't understand means losing customers (I saw it myself with some friends of mine trying to deal with WMA and AAC). MP3HD is at the moment the only lossless solution that give a sound (even a nice one at 256 kbps) working on all DAP players, DVD players and software players in the world. Without installing anything. That would make a big difference for several people on earth.
What's the most attractive in your opinion:
"Buy HD MUSIC. Cristal clear sound. Compatible with every player"
or
"Buy lossless music. CD Quality. Download the MP3 codec, the GUI and follow our two page tutorial to make it work"

Note that I'm not personaly fond of this format (crappy IDTag solution) but I'm just trying to show that the concept isn't useless.
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 18:03
Post #19





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Peter @ Mar 20 2009, 17:34) *
[*]Size of ID3v2 tags is limited to 256MB by specifications; as a result, lossless part of an mp3hd file can't be larger than 256MB.


I suspected it when the encoder returned an error on long files (55 minutes of low bitrate classical music was working but not 40 minutes of high bitrate rock music).

This post has been edited by guruboolez: Mar 20 2009, 18:04
Go to the top of the page
+Quote Post
menno
post Mar 20 2009, 18:04
Post #20


Nero MPEG4 developer


Group: Developer (Donating)
Posts: 1218
Joined: 11-October 01
From: LA
Member No.: 267



QUOTE (rpp3po @ Mar 20 2009, 08:47) *
QUOTE (Peter @ Mar 20 2009, 17:34) *
Useless format


Totally agree. Not having separate correction files is pretty bad, anyway. With mp3hd you can fit just one album on a CDR for your car stereo in the worst case. That's not the point of mp3.


While I agree on the uselessness of putting the data in ID3 tags, I disagree about the file size argument. You can remove the correction data by removing 1 frame from the ID3v2 tag, this is much faster than re-encoding your lossless files, and you will just end up with a normal "small" mp3 file. The idea is to have a lossless + lossy file at a smaller datarate than a separate lossless and lossy file, without the need to have to encode your lossless local files for portable usage.

QUOTE
I would propose a generic correction file creator for all lossy formats ("DeltaFlac"? rolleyes.gif). A very simple piece of software or plugin could decode a lossy stream (including gap correction) and generate a delta stream against the original PCM, which would then be FLAC compressed and stored somewhere safe (for the reconversion of a whole collection in 2025 to Super-AAC2-Pro-3000).

A realtime playback filter, e. g., for Foobar could just reverse the chain. Delta and lossy file could be matched by a simple tag for the case that filenames get messed up.


Lossy formats are hardly ever decoded deterministically, so your idea will never work (at least not losslessly). In SLS (so also mp3HD I assume) the original AAC/MP3 stream is not decoded by a normal lossy decoder.
Go to the top of the page
+Quote Post
callisto
post Mar 20 2009, 18:10
Post #21





Group: Members
Posts: 362
Joined: 2-September 06
From: Berlin, Germany
Member No.: 34723



@guruboolez
Ok, I get your point...
But still think it's crap. And to have a huge file on your DAP where you just can play the lossy part is just silly^^


I stick with buying CDs that I can touch, browse the booklet and look at in my cd-shelf wink.gif

This post has been edited by callisto: Mar 20 2009, 18:11


--------------------
fb2k on OSX: flac q8 > rockboxed Sansa e280v1: Vorbis q5.0
Go to the top of the page
+Quote Post
menno
post Mar 20 2009, 18:14
Post #22


Nero MPEG4 developer


Group: Developer (Donating)
Posts: 1218
Joined: 11-October 01
From: LA
Member No.: 267



QUOTE (callisto @ Mar 20 2009, 09:10) *
@guruboolez
Ok, I get your point...
But still think it's crap. And to have a huge file on your DAP where you just can play the lossy part is just silly^^

You're supposed to remove the lossless correction part before copying the file to your DAP. Of course this requires knowledge from the user about the format and I think that's where it will go wrong in general.
Go to the top of the page
+Quote Post
guruboolez
post Mar 20 2009, 18:20
Post #23





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (callisto @ Mar 20 2009, 18:10) *
And to have a huge file on your DAP where you just can play the lossy part is just silly^^

I understand your point but I don't think it's silly. Nowadays cheapest flash-memory based MP3 players are rather big (2 GB at least). People could store 4 albums (sometimes more), or 60 tracks. It's a honest performance. I also know people unable to fill their players with lossy stuff and even other who consider as silly to put so many music on a portable player.
I tend myself to maximize the capacity of my own players (including a 160GB iPod filled with 130 kbps music) but I learnt that many people acting differently.
Go to the top of the page
+Quote Post
Brent
post Mar 20 2009, 19:18
Post #24





Group: Members
Posts: 143
Joined: 27-August 07
Member No.: 46544



QUOTE (rpp3po @ Mar 20 2009, 17:47) *
I would propose a generic correction file creator for all lossy formats ("DeltaFlac"? rolleyes.gif). A very simple piece of software or plugin could decode a lossy stream (including gap correction) and generate a delta stream against the original PCM, which would then be FLAC compressed and stored somewhere safe (for the reconversion of a whole collection in 2025 to Super-AAC2-Pro-3000).

Hmm, interesting. Have a regular <filename>.mp3, and an accompanying <filename>.dflac file. Old players, just use the .mp3, new players the .dflac, and takes it's metadata from the /mp3 and uses it as a base to perform the corrections on, provided in the .dflac.

Could be quite cool. And would be _very_ convenient for me.
Go to the top of the page
+Quote Post
lvqcl
post Mar 20 2009, 19:34
Post #25





Group: Developer
Posts: 3330
Joined: 2-December 07
Member No.: 49183



some test files encoded with flac -8: 477 MB, or 976kbps. Musepack --extreme: 102 MB (210 kbps); difference between lossless and lossy encoded with flac -8: 400 MB, or 819 kbps.
So, we can make MusepackHD! laugh.gif
Go to the top of the page
+Quote Post

3 Pages V   1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th July 2014 - 20:38