IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
FAAC lower quality per bitrate than MP3lame?
F J Walter
post Nov 8 2007, 06:41
Post #1





Group: Members
Posts: 6
Joined: 18-October 07
Member No.: 47949



I have done some personal testing which is ABX style where I put the control and the compressed version into a playlist, hide the playlist window, then I listen to the first one and the second one, then shuffle and see if I can identify whether the next one is the control or the compressed version.

But I have found some strange results and that is that I am able to consistently identify the AAC at -q 120 and above, whereas I can't reliably identify (ABX) the MP3 LAME encodings at q 6 and below. Q 6 is actually a much lower bitrate than -q 120 in FAAC! So if you believe my results, you may conclude that LAME MP3 is more efficient in a perceived quality per bitrate than FAAC, which is AAC-LC!

I am doing the encoding with ffmpeg, using the latest snapshot libraries of libmp3lame and libfaac which I compiled recently. The reason for using ffmpeg is twofold - first, I can encode such that the only variable is the audio codec; the container format is the same. Second, because the whole purpose behind me doing this is figuring out a format to use as audio when encoding videos, (some of which are music videos, so I want good quality).

Is it possible that MP3 is just more transparent to me in general? Or that LAME is a lot more efficient than FAAC? Or am I not hearing artifacts in MP3 because they are masked due to the nature of the source material, and with other material I might have different experiences? The source material is an acoustic band, with a male vocalist, doing a slow ballad - quite a good recording I think.

Command lines (typical)
ffmpeg -y -i testsong.wav -ss 9.5 -t 25 -vn -acodec libfaac -aq 120 out.a120.mp4
ffmpeg -y -i testsong.wav -ss 9.5 -t 25 -vn -acodec libmp3lame -aq 6 out.m6.mp4

1st line generates 435KB file, 2nd line generates 361KB file, plus I generated a control file which copies the raw audio.
Go to the top of the page
+Quote Post
F J Walter
post Nov 8 2007, 08:15
Post #2





Group: Members
Posts: 6
Joined: 18-October 07
Member No.: 47949



Hmmm, after looking at some published tests, I see that my results are not so strange after all.

http://www.hydrogenaudio.org/forums/index....showtopic=38792
Shows insignificant difference between MP3 LAME and iTunes AAC on most sources

http://www.rjamorim.com/test/multiformat128/results.html
Shows the same

http://www.rjamorim.com/test/aac128v2/results.html
Shows that iTunes AAC to be significantly better than FAAC.

Therefore from the above, one could infer that since iTunes AAC performed signicantly better than FAAC, and iTunes AAC did not perform significantly better than MP3 LAME, then MP3 LAME is at least as good as FAAC at a given bitrate and possibly better.

By 'not significantly better' I mean that the mean of iTunes AAC seems to be within the margin of error of MP3 LAME.
Go to the top of the page
+Quote Post
xmixahlx
post Nov 8 2007, 08:33
Post #3





Group: Members
Posts: 1394
Joined: 20-December 01
From: seattle
Member No.: 693



i'm missing the part where this is supposed to be confusing...


--------------------
RareWares/Debian :: http://www.rarewares.org/debian.html
Go to the top of the page
+Quote Post
benski
post Nov 8 2007, 15:46
Post #4


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



FAAC is generally considered to be an inferior AAC encoder. iTunes AAC is improving but is still not that great.

For state-of-the-art, you should use Nero (for VBR) or Coding Technologies (for CBR).
Go to the top of the page
+Quote Post
kornchild2002
post Nov 8 2007, 17:40
Post #5





Group: Members
Posts: 2066
Joined: 8-April 05
From: Cincinnati, OH
Member No.: 21277



QUOTE (benski @ Nov 8 2007, 08:46) *
FAAC is generally considered to be an inferior AAC encoder. iTunes AAC is improving but is still not that great.

For state-of-the-art, you should use Nero (for VBR) or Coding Technologies (for CBR).


Umm... I thought the iTunes AAC encoder was on the same par as the Nero AAC encoder. Haven't past tests shown this? I know guru completed a test not too long ago but it used the February build of Nero (the August one didn't show as good of results) and the older version of the iTunes AAC encoder (not the new one). Anyway, the February build of the Nero AAC encoder seemed to beat the iTunes AAC encoder but not by much.

My point is that I thought the iTunes AAC encoder was always regarded as being of high quality. There were even times in the past where it beat Nero AAC (again, not by much).
Go to the top of the page
+Quote Post
guruboolez
post Nov 8 2007, 19:33
Post #6





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



"iTunes AAC is improving but is still not that great."

huh.gif

Small history of listening tests on HA.org:

2003: QuickTime AAC came first (collective listening test)

http://www.rjamorim.com/test/aac128test/results.html

2003: iTunes outperformed Nero Digital on 2 consecutive listening test at 130 kbps on classical music (personal tests):
http://www.hydrogenaudio.org/forums/index....c=14091&hl=
http://www.hydrogenaudio.org/forums/index....c=16395&hl=

2004: iTunes AAC won again (collective listening test)

http://www.rjamorim.com/test/aac128v2/results.html

2004: Nero Digital clearly outperforms iTunes on classical music (12 samples, personal evaluation)

http://www.hydrogenaudio.org/forums/index....c=29924&hl=

2005: iTunes ended with a better mark than Nero, disqualified (collective listening test)

http://www.listening-tests.info/mf-128-1/results.htm

2005: iTunes clearly won over Nero Digital on 200 samples at ~130 kbps (personal listening test, classical and various music):
http://www.hydrogenaudio.org/forums/index....c=38792&hl=

2005: iTunes end with a better score than Nero Digital at ~80 kbps (25 samples, personal listening test):

http://www.hydrogenaudio.org/forums/index....c=35438&hl=

2005: iTunes outperformed Nero at 96 kbps (personal evaluation, 25 samples)

http://forum.hardware.fr/hfr/VideoSon/Trai...jet_84950_1.htm

2007: Nero Digital won over iTunes VBR at 130 kbps (85 samples, test aborted)

http://www.hydrogenaudio.org/forums/index....c=58724&hl=



I believe iTunes is anything but "not that great" and is rather a state-of-art implementation of AAC since 2003 smile.gif
Go to the top of the page
+Quote Post
F J Walter
post Nov 9 2007, 02:03
Post #7





Group: Members
Posts: 6
Joined: 18-October 07
Member No.: 47949



I believe I had just been misled by information I have read about AAC replacing MP3 as a preferred MPEG audio solution due to its increased quality per bitrate.

In reality it seems that AAC encoders are not nearly as superior to advanced MP3 encoders such as LAME as I thought they were (though I hadn't seen guruboolez's most recent test, thanks for that), and that in some cases such as FAAC (the main open source implementation) they are actually significant inferior.

As I was posting this in the AAC technical forum, I was hoping that someone could give me a likely technical explanation of why LAME MP3 performs better for me in my listening tests than FAAC even when FAAC uses a higher bitrate. IE I am curious as to why is it so - I don't fully understand the way LAME MP3 achieves such performance out of what some describe as a more simple format than AAC, and why FAAC doesn't try and implement the same sort of thing for AAC, when iTunes, and now Nero, prove that it is definitely possible.

This post has been edited by F J Walter: Nov 9 2007, 02:11
Go to the top of the page
+Quote Post
guruboolez
post Nov 9 2007, 02:16
Post #8





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



AAC has some tools and a new design which makes it technically superior to MP3. But to be superior in practice, any AAC encoder need strong tunings and development. Kenyans runner are virtually faster than me, but with a lot of training I can still beat several untrained, ISO, kenyans people wink.gif

LAME is very mature and actively developed for years. Faac isn't. I guess that porting MP3 tunings to FAAC is a very complex (maybe impossible task) and need much more work than a simple copy-and-paste.
Go to the top of the page
+Quote Post
muaddib
post Nov 9 2007, 09:12
Post #9





Group: Developer
Posts: 398
Joined: 14-October 01
Member No.: 289



One test was missed: http://www.hydrogenaudio.org/forums/index....showtopic=51171
This one was public listening test at 80kbps.
Here are the results:


This one is with all participants


This one is only with 'expert' participants (those that could find at most low anchor were removed).

Nero AAC Encoder doesn't use IS nor PNS because of these test results. So, the left most results are the results that match current Nero LC at 80kbps
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th July 2014 - 02:23