Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: AAC vs. MP3: quality at the same bitrate, transparency (Read 51550 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

AAC vs. MP3: quality at the same bitrate, transparency

compressing flacs again

Well, you can either use a lower quality in LAME or switch to AAC.  Using the Nero AAC encoder with Foobar is far easier than trying to mess with QT Pro.

AAC has better quality for the same bitrates when compared to MP3.  As a results it goes transparent at lower bitrates than MP3.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #1
AAC has better quality for the same bitrates when compared to MP3.  As a results it goes transparent at lower bitrates than MP3.

Well, that's not my experience! I did some ABX'ing not so long ago with a few of my favorite tracks with much mid-high freq content which I assume is hard for an encoder. ~128kbit Nero encoding had nasty metallic/smearing artefacts, while I didn't hear a thing in my LAME -V 5 encode.

I'm using LAME 3.98 -V 5 happily and store A LOT of music on my 32G iPhone (using foobar2000 btw). Still haven't noticed an encoding artifact, but yet I don't listen much for those anyway when I'm working
Can't wait for a HD-AAC encoder :P

AAC vs. MP3: quality at the same bitrate, transparency

Reply #2
Well, that's not my experience! I did some ABX'ing not so long ago with a few of my favorite tracks with much mid-high freq content which I assume is hard for an encoder. ~128kbit Nero encoding had nasty metallic/smearing artefacts, while I didn't hear a thing in my LAME -V 5 encode.

I'm using LAME 3.98 -V 5 happily and store A LOT of music on my 32G iPhone (using foobar2000 btw). Still haven't noticed an encoding artifact, but yet I don't listen much for those anyway when I'm working

Well it's a generalization.  In general AAC provide better audio quality than MP3 for the same bitrate.  The listening tests that have been done prove this.  This does not mean it's true in all case across all music for all people.

FWIW, I heard artifacts in general listening with LAME 3.97 -V 2 that pushed me into -V 1 where I could no longer hear them (and yes I did pass the ABX test perfectly at -V 2 and fail at -V 1 to prove it to myself).

AAC vs. MP3: quality at the same bitrate, transparency

Reply #3
Perhaps AAC has a noticeable edge at lower bitrates over mp3, but the threshold of transparency is not that different between them?

AAC vs. MP3: quality at the same bitrate, transparency

Reply #4
I couldn't find the listeningtests.info domain (if i remember correctly - maybe it was already taken down as discussed previously?) to backup the claims that AAC is far surprerior to LAME.
Can't wait for a HD-AAC encoder :P

AAC vs. MP3: quality at the same bitrate, transparency

Reply #5
Well it's a generalization.  In general AAC provide better audio quality than MP3 for the same bitrate.  The listening tests that have been done prove this.  This does not mean it's true in all case across all music for all people.


Generalization statements likes this are generally frowned upon especially since the results of the public listening tests showed that neither Nero AAC nor iTunes AAC were all that superior to Lame mp3 at the 128kbps VBR bitrate.  It was a general consensus that 128kbps VBR is just too high of a bitrate to pick out differences between encoders with non-killer samples.  That is why there is talk of a new listening test that will use 96kbps VBR as the testing bitrate.  You can feel free to talk about your experiences but generalized statements like cannot be made just from your results.  Now, if you would have come out and said "I think Nero AAC or iTunes AAC would be better than Lame mp3 as that is what my blind ABX tests show" then alright.  However, you made a blanket statement saying that AAC is better.  Which AAC?  FAAC? Certainly not.  Many blind ABX tests from not only myself but others show that FAAC does not perform as well as Lame mp3 at the 96-128kbps VBR bitrate range.

I see nothing wrong with having the OP use a lower Lame setting, throwing Nero/iTunes AAC in the mix, and having them conduct their own series of blind ABX tests.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #6
Generalization statements likes this are generally frowned upon especially since the results of the public listening tests showed that neither Nero AAC nor iTunes AAC were all that superior to Lame mp3 at the 128kbps VBR bitrate.  It was a general consensus that 128kbps VBR is just too high of a bitrate to pick out differences between encoders with non-killer samples.  That is why there is talk of a new listening test that will use 96kbps VBR as the testing bitrate.  You can feel free to talk about your experiences but generalized statements like cannot be made just from your results.  Now, if you would have come out and said "I think Nero AAC or iTunes AAC would be better than Lame mp3 as that is what my blind ABX tests show" then alright.  However, you made a blanket statement saying that AAC is better.  Which AAC?  FAAC? Certainly not.  Many blind ABX tests from not only myself but others show that FAAC does not perform as well as Lame mp3 at the 96-128kbps VBR bitrate range.

I'm a bit confused by your reply.  I made a generalized statement based off the conclusions of the various listening tests that are listed in HA's own Wiki.  You're apparently telling me that drawing any sort of generalized conclusion from the test is bad.  If that's the case, then what exactly is the point posting and cataloging the tests?  Is it not general consensus based on the listening tests that at the same bitrate the good implementations of AAC do beat MP3?

AAC vs. MP3: quality at the same bitrate, transparency

Reply #7
You're apparently telling me that drawing any sort of generalized conclusion from the test is bad.

You said "In general AAC provide better audio quality than MP3 for the same bitrate. The listening tests that have been done prove this."


Which test prove this?  This one, for example (the most recent one at your link):  http://www.listening-tests.info/mp3-128-1/results.htm
is nowhere near that conclusive.

This one (the most recent 128 multi-format) also is a tie.
http://www.listening-tests.info/mf-128-1/results.htm
Creature of habit.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #8
The wiki draws no conclusions, so please provide links to the exact tests that support your points; otherwise retract your claims or get a warning for TOS8.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #9
I'm a bit confused by your reply.


Others have touched on the issues with your statements but you said AAC in general.  One of my issues (aside from the supplied documentation that goes against your blanket statement) is that you didn't specify which AAC encoder.  You just said AAC.  FAAC performs poorly (at least from my blind ABX tests and those of others) and I am sure I can dig up a bunch of other AAC encoders that haven't been updated in years and are worse than even the iTunes or FhG mp3 encoders.  Hence why AAC is not generally better than mp3.  Two AAC encoders are widely tests: Nero and iTunes/QuickTime.  So how can you make a generalized statement regarding AAC quality when really only two encoders have been heavily tested (at least here, maybe throw in the Coding Technologies encoder)?

AAC vs. MP3: quality at the same bitrate, transparency

Reply #10
The wiki draws no conclusions, so please provide links to the exact tests that support your points; otherwise retract your claims or get a warning for TOS8.

No problem.  I looked at the listed multi-format tests going back 5 years.  I saw 2 that showed AAC beating MP3s of a similar bitrate and 1 showing a near tie.  There was one test that showed MP3 edging AAC, but the test notes indicated that the AAC sample from Nero was not withing the targeted bitrate and was low compared to the LAME mp3.

AAC beating mp3:
Case #1
Case #2

Near tie:
Case #1

Further, I referenced the paper written by Karlheinz Brandenburg (PDF) which highlights the various improvements of AAC over MP3 which improve it's coding efficiency.

Based on the above mentioned research it didn't seem out of line to draw the conclusion that in general AAC (using the best available encoders) will offer better results than mp3 (using the best available encoders) at the same bitrate.  If this doesn't meet the criteria necessary here, I will be happy to edit my posts (assuming I can) and withdraw my statements.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #11
Others have touched on the issues with your statements but you said AAC in general.  One of my issues (aside from the supplied documentation that goes against your blanket statement) is that you didn't specify which AAC encoder.  You just said AAC.  FAAC performs poorly (at least from my blind ABX tests and those of others) and I am sure I can dig up a bunch of other AAC encoders that haven't been updated in years and are worse than even the iTunes or FhG mp3 encoders.  Hence why AAC is not generally better than mp3.  Two AAC encoders are widely tests: Nero and iTunes/QuickTime.  So how can you make a generalized statement regarding AAC quality when really only two encoders have been heavily tested (at least here, maybe throw in the Coding Technologies encoder)?

In my first post in the thread I did reference the Nero AAC encoder.  My following posts were in reference to that though I didn't call it out explicitly.  There are good and bad examples of all codecs.  I assumed (I know dangerous...) that we were comparing the better versions of each.  If we're going to hold FAAC against AAC shouldn't we also hold Xing against mp3.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #12
I'm not exactly satisfied.

The test you incorrectly interpret as a "near tie" is not a near tie, it is a tie.

Where are your test results showing that AAC achieves transparency at lower bitrates than mp3?

Further, I referenced the paper written by Karlheinz Brandenburg (PDF) which highlights the various improvements of AAC over MP3 which improve it's coding efficiency.

Whether these improvements have been fully realized boils down to tuning, BTW.

Do you have any personal results with the most recent encoders?

AAC vs. MP3: quality at the same bitrate, transparency

Reply #13
AAC beating mp3:
Case #1
Case #2


Not to step on anyone's toes but those were personal tests from guru.  Although I appreciate their work every time they post listening tests results, they aren't representative of a public listening test (unless the second is a public listening test, even then it appears that Lame mp3 pulls ahead but I can't read through the results as I don't understand the language being used).  Additionally, guru's first test shows that Lame at 128kbps (which was the high anchor) achieved transparency more so than Nero or iTunes AAC at 80kbps.  Rightfully so, that was the high anchor.  However, that also goes against your claims that "AAC" is able to provide transparency over Lame at lower bitrates.  Again, I don't think anyway can draw broad conclusions from both of those tests as they were personal.  guru simply shaired his results for people to use for their personal judgments, not to draw blanket statements from.

Lastly, I am not trying to hold FAAC against AAC in general.  However, when you make blanket statements regarding AAC, people are going to infer whatever they want.  You need to be specific here especially since the OP needs proper guidance.  Saying that AAC is generally better than mp3 doesn't mean anything.  Which encoders are you talking about?  Which settings were you comparing?  What about HE-AAC, that is part of AAC in general isn't it?  All sorts of questions and inferences can be made when making general statements.  Hence why you should have stated "my personal listening tests show that Nero AAC is better than Lame mp3 for me when comparing the -q0.4 and -V 5 settings [or whatever] respectively."  That would have been alright as you were informing the OP of your personal results instead of trying to make these blanket "matter of fact" statements.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #14
Where are your test results showing that AAC achieves transparency at lower bitrates than mp3?

I don't have any.  I've never seen the transparency point of a codec put to any sort of public listening test.  I think it would be hard to test unless you gave each user a slider and let them decide where each codec goes transparent and then looked for statistical trends in the data on where each goes transparent in comparison to each other.  Regardless, wouldn't achieving transparency at a lower bitrate be a natural result if a codec offered superior efficiency and better performance at the same bitrates?

Quote
Do you have any personal results with the most recent encoders?

I haven't done any substantial ABX testing with Nero 1.5.x yet.  In my really quick listening it seemed transparent at q=0.50 where as in my past ABX testing LAME 3.97 was transparent at -V 1 (and was not at -V 2) with headphones.  I never seriously ABX'd LAME 3.98.2.  I plan to do some ABX testing with Nero 1.5.x to make sure encoding q=0.50 is ample for my iPod listening.

One serious question I have is how the quality of the listener's audio setup is taken into account in these public tests?  How can you tell if differences in the samples are not audible to the listener, or simply not revealable on the listener's audio gear?

As an example, with my Sony MDR-V6 headphones LAME 3.97 at -V 1 was transparent to me.  However, when I did some other ABX testing on my main stereo several years ago when I lived in an apartment I couldn't get any mp3 to reach transparency (regardless of the compression settings) when the recordings had real depth to their soundstage (most don't) because the mp3 destroyed the depth found in the recording and made the ABX test trivial.  I repeated the test with the same tracks more recently (with all the same gear) in a different room (now in my house) and I could not pass the ABX test and identify the mp3s like I had before.  Unfortunately, the acoustics of the room in my house are not nearly as good as the room in the apartment and I do not get the depth to the soundstage that I did in my apartment.  Two different rigs, and one of them in two different rooms and I have different results.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #15
Lossy compression works best on a system/playback environment that provides a flat frequency response.

I'll be patiently awaiting the logs from your ABX results.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #16
I wouldn't hold your breath.  Thanks for splitting this from the other topic though, that way we can keep the two separate.


AAC vs. MP3: quality at the same bitrate, transparency

Reply #18
Just a little bit  I don't pretend to be perfect and we all make mistakes.  I have been heavily visiting HA for almost 7 years now (I remember reading about Lame 3.90.3, EAC, and the --alt-preset standard back when I purchased my 40GB 3G iPod in 2003 here at HA) and still make mistakes.  However, things start getting curious when people don't provide any backup and the information that they do provide is either filled with flaws, has been disputed, or represents something different (like a private listening test from a respected member here).

AAC vs. MP3: quality at the same bitrate, transparency

Reply #19
I'll be patiently awaiting the logs from your ABX results.

Sorry, but I didn't save them from years ago.  I didn't realize I would be called on the carpet on an internet forum over 3 years later to prove myself.

I can see where this is going already.  You don't believe me.  That's OK.  You don't believe that a recording can have depth to the soundstage.  That's fine too.

However, you might be surprised what you can hear when you listen to a system with a flat frequency and more importantly a flat phase response with a good recording.  Humor me for a minute here.  If you are blindfolded in a room and someone is talking to you in the room, can you tell how far away from you they are?  Of course you can.  If you go to a concert hall and sit on the stage and listen to an orchestra you can also tell with just your ears that some instruments are closer to you an others.  Why is it so hard to believe that a stereo recording can capture the the same acoustic characteristics that your two ears can hear sitting in that room or sitting on the stage?  If microphones can record the same things that your ears can hear why don't you believe that an audio system of sufficient quality can reproduce what's recorded?

FWIW, I am not an audiophool.  I don't buy into cable voodoo magic.  I'm constantly challenging people who make frivolous claims of power cables, and all that audiophile snake oil BS.  I am a big proponent of ABX / Double Blind testing.  If you look at my posts on other audio forums you'd see I'm quite the skeptic when people make extravagant claims.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #20
It has been pointed out numerous times that combining two point sources (i.e. stereo speakers) is generally not going to realistically produce the same effect as a single point source at any arbitrary location. The appearance of "depth" is going to depend very strongly on the listening environment.

Did you ever consider the possibility that the environment in which you could differentiate high bitrate mp3 encoding was the one that was flawed, exposing artifacts that would have been masked in a more normal listening environment?

AAC vs. MP3: quality at the same bitrate, transparency

Reply #21
Sorry, but I didn't save them from years ago.  I didn't realize I would be called on the carpet on an internet forum over 3 years later to prove myself.

You weren't - you were called on the carpet to follow the terms of service for the forum.

You don't believe that a recording can have depth to the soundstage.

I didn't see greynol say that.  Did you?

However, you might be surprised what you can hear when you listen to a system with a flat frequency and more importantly a flat phase response with a good recording.

Now you're telling us greynol just doesn't have a good enough system?  The man hunted Smurfs, for christ's sake, he has a good system. 

Humor me for a minute here.  If you are blindfolded in a room and someone is talking to you in the room, can you tell how far away from you they are?  Of course you can.  If you go to a concert hall and sit on the stage and listen to an orchestra you can also tell with just your ears that some instruments are closer to you an others.  Why is it so hard to believe that a stereo recording can capture the the same acoustic characteristics that your two ears can hear sitting in that room or sitting on the stage?  If microphones can record the same things that your ears can hear why don't you believe that an audio system of sufficient quality can reproduce what's recorded?

Ignoratio elenchi. 
Irrelevant
Creature of habit.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #22
Well, that's not my experience! I did some ABX'ing not so long ago with a few of my favorite tracks with much mid-high freq content which I assume is hard for an encoder. ~128kbit Nero encoding had nasty metallic/smearing artefacts, while I didn't hear a thing in my LAME -V 5 encode.


I've noticed that Nero seems to have a smearing problem with certain tracks such as Linchpin, that loses to LAME -V5.
"I never thought I'd see this much candy in one mission!"

AAC vs. MP3: quality at the same bitrate, transparency

Reply #23
Well, that's not my experience! I did some ABX'ing not so long ago with a few of my favorite tracks with much mid-high freq content which I assume is hard for an encoder. ~128kbit Nero encoding had nasty metallic/smearing artefacts, while I didn't hear a thing in my LAME -V 5 encode.


I've noticed that Nero seems to have a smearing problem with certain tracks such as Linchpin, that loses to LAME -V5.


I can attest to that as well. LAME V5 was transparent, NeroAAC had to go to q 0.6 to reach transparency.

AAC vs. MP3: quality at the same bitrate, transparency

Reply #24
If one does not accept an answer if it hasn't been proven for all music by direct comparison one cannot accept any answer. We cannot test all tracks just to be able to judge. For sufficiently high bitrates, as mentioned, there isn't even any need to judge, both AAC and MP3 seem to be fine for most music.

Still, these questions come up, and I think they are justified. But while most tracks are transparent at common bitrates with both encoders, one encoder or format might still result in more killer samples per 1000 tracks average than the other. It is very hard to produce something conclusive in that regard. A lot more killer samples for MP3 have been reported in the past, but MP3 has also a much higher market share. But that we couldn't conclude anything, yet, doesn't mean that possibly significant differences exist. It's just pretty hard to find out without a shitload of solid, structured data to process.

From a mere technical perspective, the AAC format has a clear edge over MP3, especially better transient handling and a cleaner, simpler filter bank (MP3's is a political curiosity). Assumed that two identical teams of programmers work an identical time span, a resulting AAC encoder should definitely be the better product. But that's theory. In practice we have hundreds of man-years work in the LAME and Fraunhofer MP3 code. The same is true for the popular AAC encoders, but Apple is caring about it mainly for its own platform and there are only very few Nero developers, whose work on the codec doesn't generate additional revenue for their employer anymore. If the latter wasn't already of such high quality it would seriously be time for a new open source AAC encoder project.