IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Pre-echo: Castanets, --api vs. iTunes AAC @ 320, Interesting Results!
Cygnus X1
post Jul 29 2004, 03:02
Post #1





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



I've played around with iTunes AAC but ended up settling on ALAC, because I had this nagging feeling that I was hearing pre-echo on very sharp attacks that I did not hear with LAME. Tonight, I fired up Virtual PC (Java ABC/HR is goofy on my machine) and did two quick ABX tests using a castanets sample, encoded by LAME 3.96.1 and iTunes 4.6, both @ 320kbps CBR.

(Edit: re-ran test for 16 trials each. Not done sequentially as to avoid alpha inflation (increased Type I error rate).

The results for AAC @320:

1R = C:\Documents and Settings\Angelo\Desktop\castanets aac.wav

---------------------------------------
General Comments:

---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\Angelo\Desktop\castanets aac.wav
16 out of 16, pval < 0.001


Now for LAME:


1R = C:\Documents and Settings\Angelo\Desktop\castanets insane 320.wav

---------------------------------------
General Comments:

---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\Angelo\Desktop\castanets insane 320.wav
11 out of 16, pval = 0.105

This is VERY surprising. Actually, I almost hit significance at the .05 level and somehow lost my concentration or couldn't tell the difference anymore (just like the last time). In any case, this was much harder than the AAC test. So, what's wrong with my ears...aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

And finally, LAME vs. AAC:

1L = C:\Documents and Settings\Angelo\Desktop\castanets aac.wav

---------------------------------------
General Comments:

---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\Angelo\Desktop\castanets aac.wav
16 out of 16, pval < 0.001

Results: I could tell the AAC encoding apart from the original, p<.001. I could NOT tell LAME --insane from the original, p>.10. And finally, I could tell the LAME encoding apart from the AAC one, p<.001.

This post has been edited by Cygnus X1: Jul 29 2004, 04:07
Go to the top of the page
+Quote Post
westgroveg
post Jul 29 2004, 03:15
Post #2





Group: Members
Posts: 1235
Joined: 5-October 01
Member No.: 220



It would be more interesting if you did a 16/16 or more test to be sure Lame was really better.

QUOTE
aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

Not all MP3 encoders are created equally.
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 29 2004, 03:19
Post #3





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



QUOTE (westgroveg @ Jul 28 2004, 09:15 PM)
It would be more interesting if you did a 16/16 or more test to be sure Lame was really better.

QUOTE
aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

Not all MP3 encoders are created equally.
*



I think that can be arranged. I'm assuming you mean 16 trials of each codec compared to the reference? I also might try comparing LAME to the AAC file as well...I've already established that I can hear a difference with the AAC file, but not LAME, so that might be interesting.
Go to the top of the page
+Quote Post
westgroveg
post Jul 29 2004, 03:34
Post #4





Group: Members
Posts: 1235
Joined: 5-October 01
Member No.: 220



QUOTE (Cygnus X1 @ Jul 29 2004, 02:19 PM)
QUOTE (westgroveg @ Jul 28 2004, 09:15 PM)
It would be more interesting if you did a 16/16 or more test to be sure Lame was really better.

QUOTE
aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

Not all MP3 encoders are created equally.
*



I think that can be arranged. I'm assuming you mean 16 trials of each codec compared to the reference? I also might try comparing LAME to the AAC file as well...I've already established that I can hear a difference with the AAC file, but not LAME, so that might be interesting.
*


Yes 16 trials, I think it would also be interesting & I would apprecate it, while your at it to chuck Nero's AAC encoder & FhG fastenc -hq.
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 29 2004, 04:08
Post #5





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



Ok, have a look at my edited results...significance level increased to p<.001.

By the way, I'm on OS X and don't have a copy of Nero to run on VPC. Does the trial version of Nero come with the AAC encoder?
Go to the top of the page
+Quote Post
Garf
post Jul 29 2004, 18:38
Post #6


Server Admin


Group: Admin
Posts: 4882
Joined: 24-September 01
Member No.: 13



Yes.
Go to the top of the page
+Quote Post
Garf
post Jul 29 2004, 18:39
Post #7


Server Admin


Group: Admin
Posts: 4882
Joined: 24-September 01
Member No.: 13



QUOTE (Cygnus X1 @ Jul 29 2004, 04:02 AM)
So, what's wrong with my ears...aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?


MP3 encoders, notably LAME with presets, have had much more tuning than the relatively newer AAC encoders. So stuff like this can happen, but it should not be the general case.

This post has been edited by Garf: Jul 29 2004, 18:41
Go to the top of the page
+Quote Post
guruboolez
post Jul 30 2004, 01:44
Post #8





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Your results surprised me. I did some pre-echo tests in the past with QT AAC encoder, and results were really good. MP3 have inherent flaws, and honestly, I would be surprised if one day one MP3 encoder could achieve this quality, even at 640 kbps. Maybe is latest iTunes encoder broken somewhere? To be sure, I've quickly compared iTunes 4.6 AAC at 320 kbps to lame 3.97a3 at the same bitrate.

CODE
foo_abx v1.2 report
foobar2000 v0.8.3
2004/07/30 02:31:34

File A: file://C:\ABX\castanets-lame3.97a3-320.mp3
File B: file://C:\ABX\castanets-iTunes320.m4a

02:31:35 : Test started.
02:32:00 : 01/01  50.0%
02:32:08 : 02/02  25.0%
02:32:16 : 03/03  12.5%
02:32:24 : 04/04  6.3%
02:32:31 : 05/05  3.1%
02:32:43 : 06/06  1.6%
02:32:50 : 07/07  0.8%
02:32:54 : 08/08  0.4%
02:32:58 : 09/09  0.2%
02:33:02 : 10/10  0.1%
02:33:07 : 11/11  0.0%
02:33:11 : 12/12  0.0%
02:33:15 : 13/13  0.0%
02:33:20 : 14/14  0.0%
02:33:24 : 15/15  0.0%
02:33:29 : 16/16  0.0%
02:33:33 : 17/17  0.0%
02:33:38 : 18/18  0.0%
02:33:42 : 19/19  0.0%
02:33:47 : 20/20  0.0%
02:33:49 : Test finished.

----------
Total: 20/20 (0.0%)


This test was very easy. The first file (MP3) has serious smearing (first three attacks - the rest is less disturbing). The second one sounds natural to my ears. I'm not sure that I can ABX it against the reference (I'm too tired now). This AAC encoding was very sharp, and I don't ear any other problems.
Go to the top of the page
+Quote Post
westgroveg
post Jul 30 2004, 02:54
Post #9





Group: Members
Posts: 1235
Joined: 5-October 01
Member No.: 220



guruboolez, the best way to compare your results with Cygnus would have been to use lame 3.96 not an alpha.

This post has been edited by westgroveg: Jul 30 2004, 02:55
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 30 2004, 02:55
Post #10





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



QUOTE (guruboolez @ Jul 29 2004, 07:44 PM)
Maybe is latest iTunes encoder broken somewhere? To be sure, I've quickly compared iTunes 4.6 AAC at 320 kbps to lame 3.97a3 at the same bitrate.


*


I think that's quite possible. You're using QT 6.5 to encode, right? Also, I used LAME 3.96 as opposed to 3.97a3. I will upload my samples for you...see if my 320kbps AAC file sounds different.

What I was hearing in the AAC file was a smearing/combining of a few "clicks" after the 2.5 second mark, along with a sort of flanging that went from L to R, whereas the original and MP3 file seemed to emanate from the "front" of the channel and the smearing was harder to recognize.
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 30 2004, 03:06
Post #11





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



I think I just solved the mystery, and the news is not good.

I was just playing around with samples, using QT instead of iTunes. When played through QT itself, the AAC file exhibits NO artifacts whatsoever. However, for the ABX test, I decoded the AAC back to wav in iTunes. The wav file decoded by iTunes has artifacts that the AAC file does not!!

Worse still, I have AAC files encoded with PsyTel --extreme that play back as though half the frames are missing! They play back fine in Real. Let me run a quick ABX test to make sure that the decoded file and the AAC file really do sound different. If so, this means that iTunes has some problems with AAC decoding! sad.gif
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 30 2004, 03:16
Post #12





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



Sure enough, when decoded using iTunes 4.6, the 320kbps AAC file has artifacts that give it away every time, and the one decoded through QT sounds perfect.

Sample A: castanets aac.wav
Sample B: castanets aac iTunes.wav

1 of 1, p = 0.5
2 of 2, p = 0.25
3 of 3, p = 0.125
4 of 4, p = 0.062
5 of 5, p = 0.031
6 of 6, p = 0.015
7 of 7, p = 0.0070
8 of 8, p = 0.0030
9 of 9, p = 0.0010
10 of 10, p = <.001
11 of 11, p = <.001
12 of 12, p = <.001
13 of 13, p = <.001
14 of 14, p = <.001
15 of 15, p = <.001
16 of 16, p = <.001

Looks like I just found a bug in iTunes dry.gif
Go to the top of the page
+Quote Post
guruboolez
post Jul 30 2004, 04:12
Post #13





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Can't sleep dry.gif
I could confirm the artifact (something electric/metallic in the middle of the castenet's bunch: very obvious). More strangely, the first four attacks (and not three as I said previously) are also unsharp compared to faad2 decoding (foobar2000). Something like pre-echo/hollow sound. It's not very pronounced, but not very hard to hear. I'll ABX it tomorrow to be really sure (I'm between reality and dream on the moment wink.gif).
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 30 2004, 04:41
Post #14





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



QUOTE (guruboolez @ Jul 29 2004, 10:12 PM)
I'll ABX it tomorrow to be really sure (I'm between reality and dream on the moment wink.gif).
*


Hopefully, you'll not hear castanets in your dreams laugh.gif I myself have to be especially careful about what I listen to before bedtime!
Go to the top of the page
+Quote Post
guruboolez
post Jul 30 2004, 12:39
Post #15





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



It wasn't a hallucination. iTunes decoding has less sharpness than faad2 on the first four attacks of castanets.wav sample. Difference is in my opinon realy obvious (90 seconds only for a 21/21 score - listening range was 0.0 - 1.5). It's the first time I can hear pre-echo introduced by a DEcoder.
Did someone (Menno wink.gif) have an explanation? TNS decoding?


CODE
foo_abx v1.2 report
foobar2000 v0.8.3
2004/07/30 13:32:49

File A: file://C:\ABX\castanets-iTunes320.m4a
File B: file://C:\ABX\castanets-iTunes_decoding.wav

13:32:51 : Test started.
13:32:56 : 01/01  50.0%
13:33:00 : 02/02  25.0%
13:33:04 : 03/03  12.5%
13:33:08 : 04/04  6.3%
13:33:11 : 05/05  3.1%
13:33:16 : 06/06  1.6%
13:33:20 : 07/07  0.8%
13:33:24 : 08/08  0.4%
13:33:27 : 09/09  0.2%
13:33:32 : 10/10  0.1%
13:33:36 : 11/11  0.0%
13:33:39 : 12/12  0.0%
13:33:43 : 13/13  0.0%
13:33:46 : 14/14  0.0%
13:33:50 : 15/15  0.0%
13:33:54 : 16/16  0.0%
13:33:57 : 17/17  0.0%
13:34:01 : 18/18  0.0%
13:34:05 : 19/19  0.0%
13:34:09 : 20/20  0.0%
13:34:14 : 21/21  0.0%
13:34:15 : Test finished.

----------
Total: 21/21 (0.0%)
Go to the top of the page
+Quote Post
guruboolez
post Jul 30 2004, 12:50
Post #16





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



People might be interested to note that iTunes is not alone to suffer from this problem. I've just tried on a fresh installed Winamp 5.04 (no additionnal component like faad2), and the castanet.mp4 file exhibits the same problems (lack of sharpness, artifact ["clicks"] in the middle).
More strangely, if the same file is demuxed in a raw .aac form, Winamp 5.04 could play it flawlessly.

Can just say that: many thanks to menno for his great job on faad2.1 smile.gif
Go to the top of the page
+Quote Post
Garf
post Jul 30 2004, 13:20
Post #17


Server Admin


Group: Admin
Posts: 4882
Joined: 24-September 01
Member No.: 13



Apple's encoder is, IIRC, originally based on Dolby's. Likely, their decoder is also. The Dolby decoder has known bugs with TNS, and is used in Winamp too, when decoding .MP4.

When winamp decodes .AAC's, it uses the CodingTech decoder which does not have this bug. FAAD does not have this bug either.

This post has been edited by Garf: Jul 30 2004, 13:23
Go to the top of the page
+Quote Post
menno
post Jul 30 2004, 14:37
Post #18


Nero MPEG4 developer


Group: Developer (Donating)
Posts: 1218
Joined: 11-October 01
From: LA
Member No.: 267



Confirmed that iTunes does not produce proper output. It doesn't look like the same bug as the Dolby decoder in Winamp has. The problem here seems to be in the frequencies below 1000 Hz.

Menno
Go to the top of the page
+Quote Post
Cygnus X1
post Jul 30 2004, 15:14
Post #19





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



So, it sounds like the pre-echo problems I thought were attributed to an encoder flaw turned out to be a decoder problem, NOT a problem with QT AAC at 320kbps. I wonder if this occurs at other bitrates as well?

Later today, I can conduct more ABX tests at different bitrates, comparing QT decoded AAC to that of iTunes. Let me know if you think this would be helpful.

This post has been edited by Cygnus X1: Jul 30 2004, 15:15
Go to the top of the page
+Quote Post
loophole
post Aug 27 2004, 13:56
Post #20





Group: Members
Posts: 273
Joined: 18-June 03
Member No.: 7254



It seems strange that QT would use a different decoder to iTunes...
Go to the top of the page
+Quote Post
wkwai
post Aug 29 2004, 13:39
Post #21


MPEG4 AAC developer


Group: Developer
Posts: 398
Joined: 1-June 03
Member No.: 6943



QUOTE (menno @ Jul 30 2004, 05:37 AM)
Confirmed that iTunes does not produce proper output. It doesn't look like the same bug as the Dolby decoder in Winamp has. The problem here seems to be in the frequencies below 1000 Hz.

Menno
*



Have you try conducting conformance tests on the iTunes decoder?

wkwai
Go to the top of the page
+Quote Post
Cygnus X1
post Aug 29 2004, 15:00
Post #22





Group: Members (Donating)
Posts: 676
Joined: 5-June 02
From: New York
Member No.: 2224



I recently did a clean install of OS X 10.3 in order to go back to QT 6.4 and iTunes 4.2....the decoder problems are gone. Both iTunes encoded files and PsyTel VBR play back without any of the very obvious noise and glitches that I saw in 6.5.1/ iTunes 4.6. I am afraid of upgrading either iTunes or QT until Apple comes out with a version that corrects the decoding AND encoding glitches seen in the latest version.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th July 2014 - 21:59