IPB

Welcome Guest ( Log In | Register )

MPC vs VORBIS vs MP3 vs AAC at 180 kbps, 2nd checkup with classical music
guruboolez
post Aug 21 2005, 19:33
Post #1





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Preliminary notes


Last summer I performed a blind listening comparison between three different audio formats, all set for ~175 kbps encodings. The purpose of the test was to investigate about encoding quality with classical music (and only classical) and to see which format would be the most efficient (i.e. the closest to transparency at lowest bitrate possible) for this kind of music. As jumping-off place for bitrate I took MPC –standard preset which was indisputably recognized as the best encoding solution outputing at 175...190 kbps on average. And indeed, the test ended on musepack superiority. MPC was even superior to Vorbis and MP3 at presets presenting higher bitrate (~195 kbps for LAME, ~185 for Vorbis against ~175 for musepack). Consequently, MPC encodings appeared to sound better and to be smaller at the same time. Amazed by the existent gap between all contenders I conclude my specific test with these words: “I didn’t think that MPC –standard was so in advance”.

My vacation are now quite over. I performed during my free time a big checkup of lossy quality at 80 kbps and 96 kbps (this one has to be translated in english dry.gif ), and it’s too late now to complete the 128 kbps I planned to do in the same silly conditions (150+35 tested samples). But I used my small remaining time to do again the listening test at 175 kbps I did last year, with the same 18 samples and the same hardware.



Why doing again the same test?


As a result of constant evolution of most audio encoders I consider my previous results as really outdated. I recall that Vorbis encodings were done with MEGAMIX I (hybrid encoder melting aoTuV beta 2, Garf Tuned 2 and Quantum Knot tunings). This encoder didn't subsist for a long time... and doesn’t exist anymore; it was replaced by MEGAMIX II, then official 1.1 with Impulse Trigger Profile + Impulse Noisetune switches, which was finally followed by aoTuV beta 3 and beta 4. The same goes for LAME: 3.97 alpha 3 was tested, and during this time LAME developers have submitted eight new versions of this alpha and a few other ones (lame_may, lame_june...)! MPC has also changed: from 1.14 beta to 1.15 alpha which is now considered as safe to use.
As a consequence of this evolution, problems audible last years (kind of ringing for LAME, or noise and coarseness for Vorbis) may be corrected or at least be lowered. The first purpose of my test is therefore to check the outcomes of recent tunings done for high bitrate settings.

There’s also a second point which stimulated me to do again the test and this point is called AAC. I haven’t tested AAC last year for technical and moral reasons. Technically, iTunes encoder couldn’t be set to ~175 kbps; Apple's AAC encoder wasn't also gapless and is for my purpose unsuitable for my conception of artefact-free encodings. I also felt as dishonest the inclusion of Nero AAC: it had recognized issues with classical first and a new encoder supposed to solve these problems was announced as imminent. Some readers suggest me to include faac as competitor, but I felt as unfair to test an encoder which was probably not the state of the art of AAC format and to compare it to the most advanced implementation of other formats (MEGAMIX and LAME 3.97).
I never regret my choice. But this absence of AAC frustrated my curiosity for a long time, because I had strictly no idea about comparative performance of this format with other contenders. That’s why I decided to absolutely include AAC this time. WMAPro will also be tested this time if possible.

The purpose of my test is therefore to obtain a fresh photography of the current performance of all modern lossy formats with classical music using the most advanced implementations for each of them.


I. Choosing the encoders


My purpose being to test most advanced encoders the choice of format hasn't to be controversial for most of them:

MP3: LAME 3.97 alpha 11. Release date: July 2005. Note: --vbr-new encoding mode.
MPC: mppenc 1.15v. Release date: march 2005.
Vorbis: aoTuV beta 4. Release date: June 2005, updated in July 2005 (merged with SVN 1.1.1).
WMAPro: no choice here: it's 9.1 or nothing. Release date: during 2004.

Choosing the good AAC encoder is much harder:
Apple AAC: There's still no VBR mode with iTunes. Consequently it's currently impossible to use Apple's AAC encoder unless other contenders will output an average bitrate close to either 160 kbps or 192 kbps. It's unlikely...
Nero Digital AAC: the most advanced VBR AAC encoder and therefore better placed to represent the AAC format. Big problem: should I use the 'high' and defaulted encoder or rather the 'fast' one which is really better at lower bitrate with classical music? The first one is still recommended by all Nero's developers and it's a valid reason to choose it instead of something they don't consider as stable enough (Garf, JohnV and Ivan Dimokovic). But the situation has maybe changed since their recommendation; I wouldn't also discard too quickly the possibility of using an encoder working better for the difficulties proper to the musical genre I'll test. The debate could be endless if a trivial but objective argument hadn't close the debate: the average bitrate of VBR mode of both encoders (see below).
faac AAC: testing faac might also be interesting. And even for fun, it would give me the possibility to oppose four different open-source implementations of four different formats smile.gif But such friendly comparison has a price: increasing the onerousness of the test which is anything but easy at this bitrate...


II. Targeting a bitrate


The purpose of my test is not to see what encoders could do with xxx kbps for each sample; I don't plan to force each encoding reaching a precise bitrate. My purpose is to stay close to the real usage of a vast majority of listeners (if not all...): using for every encoding one fixed setting which should statistically corresponds on average to the desired bitrate. That's why it's really fundamental to precisely know the average bitrate corresponding to a defined preset. And there's only one way to get it: encoding several tracks or albums.
Last year, I used as reference ~20 classical (+3 non-classical) albums. This year, I decided to be more methodical. I’m now using 150 different tracks (I mean full tracks) coming from 150 different CD in order to increase the variety of encoded tracks. It’s important to note that I didn’t choose randomly those tracks. I meticulously worked to get a representative microcosm of my full classical library, balanced between different grand ensemble (vocal, orchestral, chamber, soloist recording). This collection is nothing more than the 150 full tracks from which I’ve extracted 150 short samples in order to build a “catalogue raisonné” of musical situations occurring with classical music (see this test).

I genuinely expect from this methodically constructed library to be a highly representative panel of my classical collection. My assumption could be verified by checking the average bitrate of the entire collection encoded with WavPack -fx5 (all my >1000 CD digital library is encoded with this preset): 642 kbps for the selection of 150 tracks against 635 kbps for a complete set of more than 15000 tracks. The deviation is inferior to 1%! blink.gif Statistics are really magical.


III. Observing bitrates


I started with MPC which must give the reference bitrate. All other competitors have to be set in order to get a similar value.


MPC: --quality 5 corresponds precisely to 184,54 kbps. This is higher from what I expected first (~175 kbps). The 150 reference tracks are maybe not as representative as supposed. I also tried 1.14 (used last year) with the same preset and --xlevel: 176,28 kbps, much closer to the native average bitrate of --standard profile and reassuring me about the representativity of my collection of tracks. The bitrate has therefore inflated by 4.7% from 1.14 to 1.15v with classical.
=> I'll therefore try to get from all other encoders a setting which outputs to 184,5 kbps ±2% (180,5...188,1 kbps).

MP3: I first tried -V2 --vbr-new, which corresponds to the former --preset fast standard. Average bitrate is 181,79 kbps. Now, this value is lower from what I estimated last year (and that's why I tested -V3 in addition to -V2)... Indeed, 3.97alpha3 -V2 would output to 192,99 kbps. Nice gain (-5.80%). Obviously LAME developpers also worked on efficiency. Gain is great enough that LAME --preset standard could now be fairly compared to MPC --standard. But I recall another time that it only applies for classical (I suppose that bitrate is higher with other musical suffering from sb21 issue).

Vorbis: aoTuV beta 4 -q6,00 leads to 181,48 kbps. This is lower than what I expected, and it's also lower than MPC --standard bitrate. I get 186,99 kbps for the old MEGAMIX I. Bitrate has therefore be lowered with latest aoTuV.
-q6,00 could therefore be directly compared to MPC --standard and LAME --preset fast standard (for classical music).

WMAPro: VBR75 leads to 150,24 kbps. The next available preset is VBR90 and it leads to 203,96 kbps. Both are very far for the range I fixed and consequently WMAPro can't compete in this test.

Nero Digital AAC: Like LAME and WMAPro Nero Digital doesn't offer any precise VBR scale but seven presets. -internet leads to ~142 kbps for both 'high' and 'fast' encoders. -streaming high corresponds to 176,14 kbps and -streaming fast to 193,33 kbps. Consequently none of them is inside the fixed range; the closest one is -streaming high and is therefore the less unacceptable solution (I recall that the 'high' encoder is still the recommended one).

faac AAC: this is the only encoder able to fit into the fixed bitrate range (thanks to the precise VBR scale alla vorbis & mpc). AAC faac –q 175 leads to 180,92 kbps. This –q setting won’t probably correspond to 180 kbps with other musical genre and that’s the occasion to recall another time that the whole test is specific to classical music and nothing else.


Recapitulative table

CODE
         bitrate_2004   bitrate_2005     evolution in kbps   ...in %

MPC          176,28         184,54            +8,26 kbps      +4,69 %
MP3          192,99         181,79           -11,20 kbps      -5,80 %  
Vorbis       186,99         181,48            -5,51 kbps      -2,95 %
AAC faac   not tested       180,92              --              --
AAC Nero   not tested       176,14              --              --


=> faac, LAME, aoTuV are very close each others (difference is inferior to 0,9 kbps!). MPC presents a higher bitrate (+3 kbps) and Nero Digital a lower one (-5 kbps). The gap between the extreme is worrying: approximately 5% corresponding to 8 kbps. That's not a huge difference but these eight missing kbps may lead to a significant difference in quality. I could discard Nero Digital for this test but I would consider this choice as a mistake. For my own curiosity I'm also very impatient to see how would perform an advanced implementation of AAC in comparison to other formats, even if bitrate are not fully comparable.

=> As a consequence I decided to test both Nero Digital AAC and faac AAC, and I will consider Nero Digital presence as a "bonus" interesting to watch rather than an entire competitor. That's why my final diagramme (plots) will graphically separate Nero AAC results from other contenders. I hope this will avoid unecessary debate about any kind of unfairness based on bitrate disparity.



SUMMARY

Are going to be test:

AAC: faac 1.24.1. Release date: end 2004 (?). Setting: -q175
AAC: Nero Digital aacenc32 v.3.2.0.15. Release date: June 2005. Setting: -streaming (high/default encoder).
MP3: LAME 3.97 alpha 11. Release date: July 2005. Setting: -V2 --vbr-new
MPC: mppenc 1.15v. Release date: march 2005. Setting: --quality 5
Vorbis: aoTuV beta 4 based on 1.1.1. Release date: July 2005. Setting: -q6,00



IV. Additional information


I performed all my last listening tests on a Creative Audigy 2 soundcard, which resamples everything to 48000 KHz. Some people consider that internal resampling (transparent in my opinion) is treating unfairly musepack and would biased any listening test. To cut the controversial short, I installed my (better) Terratec DMX6Fire 24/96 which doesn't resample 44100 KHz files (I'm not using it anymore for daily listening because of interference with my VIA chipset).

HARDWARE & SOFTWARE SETTINGS:

soundcard: Terratec DMX6Fire 24/96
headphone: BeyerDynamic DT-531
amp: Onkyo MT-5
software player: Java ABC/HR 0.5 beta 5.
software decoder: foobar2000 0.83 (in order to automatically get files free of offset and to solve my incompatibility issues occuring with Vorbis).

TESTING PRINCIPLES:

ABX phase : To limit the listening fatigue and to end the test before I left my appartment, I restricted the ABX tests to the most transparent encodings (note > 4.00).
Number of trials : eight trials as a minimum. I recall that schnofler's ABC/HR software doesn't reveal to score until the test is closed by the user (and it also can't be resume). Therefore the number of trails hasn't to be fixed: as long as score is hidden the pval isn't ruined. That's why I add more trials when I suspect bad results. I never exceed 16 trials: if something is really transparent I didn't persecute the encoding smile.gif
Notation : My notation was very severe last year, with a full dynamic range of notation (a lot of notes were inferior to 2.0). That's why I decided to add 10 points to each score (in order to disconnect the notation from the usual corresponding scale). This year, I tried to respect the ITU scale. When a difference is audible but not really annoying, the notation is at least equal to 4.0 and my hairs must stand on end to allow a notation inferior to 2.0 (from "annoying" to "very annoying"). Notation is still severe (I keep in mind that all encodings were set at 180 kbps) and that's why results I get here can't absolutely not be compared to other listening tests I done, especially those performed for low bitrate settings. By the way, there are no anchors in this test (high anchor is of course unecessary here).
Samples: Same as last year. See this thread.
Gain: I hadn't modify the gain of any file. All were played at their original volume.

This post has been edited by guruboolez: Aug 31 2005, 11:00
Go to the top of the page
+Quote Post
 
Start new topic
Replies
guruboolez
post Aug 21 2005, 19:35
Post #2





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Sample 12: Liebestod
Short description: Soprano voice and full orchestra (modern instruments) – submitted by harashin.
Possible problems: same as sample_10. Violins (in orchestra) are often difficult to encode and may introduce another encoding difficulty.
replaygain_sample_gain: -3.57 dB
(indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1L = E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC faac].wav
2L = E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MPC 1.15v].wav
3L = E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MP3 LAME].wav
4L = E:\SUMMER TESTS 2005\HQ180\12_Liebestod [vorbis aoTuV].wav
5L = E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC Nero Digital].wav

---------------------------------------
General Comments:
---------------------------------------
1L File: E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC faac].wav
1L Rating: 3.5
1L Comment: Audible distortion on voice; strings are not totally transparent neither.
---------------------------------------
2L File: E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MPC 1.15v].wav
2L Rating: 4.0
2L Comment: Irregularities audible with both orchestra and soprano. ABXed during ~13.00 - ~18.00. Voice has something hollowed, not really pleasant, but this problem is confined to short part of this sample.
---------------------------------------
5L File: E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC Nero Digital].wav
5L Rating: 3.0
5L Comment: Voice is distorted, like 1L. Most annoying are the violins (orchestra: 23.00-27.00), not really pleasant.
---------------------------------------

ABX Results:
Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MPC 1.15v].wav
7 out of 8, pval = 0.035
Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MP3 LAME].wav
8 out of 16, pval = 0.598
Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC faac].wav
8 out of 8, pval = 0.0030
Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [vorbis aoTuV].wav
10 out of 16, pval = 0.227
Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC Nero Digital].wav
8 out of 8, pval = 0.0030


---- Detailed ABX results ----
Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MPC 1.15v].wav
Playback Range: 14.167 to 20.529
1:04:38 AM p 1/1 pval = 0.5
1:04:41 AM p 2/2 pval = 0.25
1:04:45 AM p 3/3 pval = 0.125
1:04:49 AM p 4/4 pval = 0.062
1:04:55 AM p 5/5 pval = 0.031
1:04:59 AM p 6/6 pval = 0.015
1:05:04 AM p 7/7 pval = 0.0070
1:05:08 AM f 7/8 pval = 0.035

Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [MP3 LAME].wav
Playback Range: 21.919 to 29.992
1:05:43 AM f 0/1 pval = 1.0
1:05:47 AM f 0/2 pval = 1.0
1:05:52 AM p 1/3 pval = 0.875
1:05:56 AM f 1/4 pval = 0.937
1:06:04 AM p 2/5 pval = 0.812
1:06:07 AM p 3/6 pval = 0.656
1:06:11 AM p 4/7 pval = 0.5
1:06:18 AM f 4/8 pval = 0.636
1:06:21 AM p 5/9 pval = 0.5
1:06:26 AM f 5/10 pval = 0.623
1:06:29 AM p 6/11 pval = 0.5
1:06:44 AM f 6/12 pval = 0.612
1:06:50 AM p 7/13 pval = 0.5
1:06:55 AM f 7/14 pval = 0.604
1:07:03 AM p 8/15 pval = 0.5
1:07:23 AM f 8/16 pval = 0.598

Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC faac].wav
Playback Range: 22.372 to 29.992
1:03:36 AM p 1/1 pval = 0.5
1:03:40 AM p 2/2 pval = 0.25
1:03:44 AM p 3/3 pval = 0.125
1:03:48 AM p 4/4 pval = 0.062
1:03:57 AM p 5/5 pval = 0.031
1:04:02 AM p 6/6 pval = 0.015
1:04:05 AM p 7/7 pval = 0.0070
1:04:08 AM p 8/8 pval = 0.0030

Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [vorbis aoTuV].wav
Playback Range: 21.919 to 29.992
1:07:35 AM f 0/1 pval = 1.0
1:07:41 AM p 1/2 pval = 0.75
1:07:47 AM f 1/3 pval = 0.875
Playback Range: 12.082 to 20.796
1:08:10 AM p 2/4 pval = 0.687
1:08:16 AM f 2/5 pval = 0.812
1:08:23 AM p 3/6 pval = 0.656
1:08:29 AM p 4/7 pval = 0.5
1:08:35 AM p 5/8 pval = 0.363
1:08:41 AM p 6/9 pval = 0.253
1:08:48 AM p 7/10 pval = 0.171
1:08:55 AM f 7/11 pval = 0.274
1:09:03 AM f 7/12 pval = 0.387
1:09:08 AM p 8/13 pval = 0.29
1:09:16 AM f 8/14 pval = 0.395
1:09:21 AM p 9/15 pval = 0.303
1:09:27 AM p 10/16 pval = 0.227

Original vs E:\SUMMER TESTS 2005\HQ180\12_Liebestod [AAC Nero Digital].wav
Playback Range: 17.161 to 22.186
1:09:55 AM p 1/1 pval = 0.5
1:10:00 AM p 2/2 pval = 0.25
1:10:05 AM p 3/3 pval = 0.125
1:10:10 AM p 4/4 pval = 0.062
1:10:15 AM p 5/5 pval = 0.031
1:10:19 AM p 6/6 pval = 0.015
1:10:22 AM p 7/7 pval = 0.0070
1:10:26 AM p 8/8 pval = 0.0030


Nero Digital is the worst and pay for distortions occurring in orchestral moments¹. faac offers similar performance but with less distortions on violins. MPC is better and presents some irregularities especially on voice (see sample_10 for similar problem). Note: last year I unmasked MPC on ABCHR phase but missed the ABX test. Has the encoding quality regressed or did my hearing improved a bit?
Two files are transparent to my ears (a record for this test): LAME (8/16) and aoTuV (10/16). Note: there were a clear ringing audible with alpha 3 of LAME; now it’s totally gone.

¹ I already noticed similar distortions in the past with violins/chorus with older version of Nero AAC; the new ‘fast’ encoder is used to perform better. The same problem is audible in similar conditions with sample_18.



Sample 13: LadyMacbeth
Short description: full and hysterical orchestra (modern instruments) with a lot of percussions and cymbals – submitted by harashin.
Possible problems: pre-echo is likely; distortions on other instruments (especially cymbals and brass) may also occur.
replaygain_sample_gain: -5.27 dB
(indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1R = E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [vorbis aoTuV].wav
2R = E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [AAC faac].wav
3R = E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [AAC Nero Digital].wav
4L = E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [MPC 1.15v].wav
5L = E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [MP3 LAME].wav

---------------------------------------
General Comments:
---------------------------------------
1R File: E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [vorbis aoTuV].wav
1R Rating: 4.2
1R Comment: Cymbals are slightly boosted by noise. Not unpleasant, but audible on direct comparison.
---------------------------------------
2R File: E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [AAC faac].wav
2R Rating: 3.5
2R Comment: Cymbals are smeared.
Percussive instruments (11.00 - 16.00) are slightly hollowed.
---------------------------------------
3R File: E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [AAC Nero Digital].wav
3R Rating: 3.0
3R Comment: Cymbals are smeared and also distorted.
Pre-echo on percussive part.
---------------------------------------
4L File: E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [MPC 1.15v].wav
4L Rating: 4.0
4L Comment: Cymbals are distorted (they are frail and sound a bit false, like after a noise reduction process).
---------------------------------------
5L File: E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [MP3 LAME].wav
5L Rating: 3.0
5L Comment: Cymbals are distorted. Annoying.
Slight smearing on percussive segment.
---------------------------------------

ABX Results:
Original vs E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [MPC 1.15v].wav
8 out of 8, pval = 0.0030
Original vs E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [vorbis aoTuV].wav
10 out of 12, pval = 0.019


---- Detailed ABX results ----
Original vs E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [MPC 1.15v].wav
Playback Range: 04.620 to 08.096
1:32:43 AM p 1/1 pval = 0.5
1:32:50 AM p 2/2 pval = 0.25
1:32:56 AM p 3/3 pval = 0.125
1:33:05 AM p 4/4 pval = 0.062
1:33:10 AM p 5/5 pval = 0.031
1:33:13 AM p 6/6 pval = 0.015
1:33:16 AM p 7/7 pval = 0.0070
1:33:20 AM p 8/8 pval = 0.0030

Original vs E:\SUMMER TESTS 2005\HQ180\13_LadyMacbeth [vorbis aoTuV].wav
Playback Range: 04.620 to 08.096
1:33:54 AM p 1/1 pval = 0.5
1:34:05 AM f 1/2 pval = 0.75
1:34:08 AM p 2/3 pval = 0.5
1:34:11 AM p 3/4 pval = 0.312
1:34:14 AM p 4/5 pval = 0.187
1:34:17 AM p 5/6 pval = 0.109
1:34:21 AM p 6/7 pval = 0.062
1:34:28 AM p 7/8 pval = 0.035
1:34:34 AM p 8/9 pval = 0.019
1:34:38 AM p 9/10 pval = 0.01
1:34:43 AM p 10/11 pval = 0.0050
1:34:46 AM f 10/12 pval = 0.019


LAME and Nero Digital both produce the worst sounding: cymbals are distorted and smearing is also audible. Problems occurring with faac are less disturbing, despite of an unusual form of artefact (hollowed sound on drums). MPC and aoTuV are both very good. Musepack distorts cymbals which don't sound true again (see sample_01 and to a less extend sample_08) whereas Vorbis adds noise (see sample_07 and sample_11) which is less unpleasant to my ears and also harder to detect. ABX: 8/8 for MPC and 10/12 for aoTuV.


Sample 14: Vivaldi RV93
Short description: small concertino performed on period instruments.
Possible problems: on mandolin, softened or distorted harpsichord in continuo.
replaygain_sample_gain: -1.32 dB
(indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1R = E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [MP3 LAME].wav
2R = E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [AAC Nero Digital].wav
3L = E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [AAC faac].wav
4R = E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [vorbis aoTuV].wav
5R = E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [MPC 1.15v].wav

---------------------------------------
General Comments:
---------------------------------------
1R File: E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [MP3 LAME].wav
1R Rating: 3.0
1R Comment: Harpsichord is distorted, and slightly smeared (rather imprecise).
---------------------------------------
2R File: E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [AAC Nero Digital].wav
2R Rating: 3.5
2R Comment: Harpsichord is not perfect; better than previous file (more precise).
Violin suffers from distortions too [it's the only encoding in this situation].
---------------------------------------
3L File: E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [AAC faac].wav
3L Rating: 3.2
3L Comment: Slightly better than 1R, but worse than 2R (same issues).
---------------------------------------
4R File: E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [vorbis aoTuV].wav
4R Rating: 4.0
4R Comment: Violins are not transparent: they sound a bit false... can't define the problem: it's like a mix between thinness and fatness, noise reduction and additionnal noise. Weird... But it's very limited.
---------------------------------------
5L File: E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [REFERENCE].wav
5L Rating: 4.5
5L Comment: Very subtle distortion on string [ABXed with pval = 0.054...]
---------------------------------------

ABX Results:
Original vs E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [vorbis aoTuV].wav
9 out of 10, pval = 0.01
Original vs E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [MPC 1.15v].wav
8 out of 10, pval = 0.054


---- Detailed ABX results ----
Original vs E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [vorbis aoTuV].wav
Playback Range: 01.134 to 04.214
1:58:25 AM p 1/1 pval = 0.5
1:58:30 AM p 2/2 pval = 0.25
1:58:34 AM p 3/3 pval = 0.125
1:58:38 AM p 4/4 pval = 0.062
1:58:47 AM f 4/5 pval = 0.187
1:58:52 AM p 5/6 pval = 0.109
1:58:55 AM p 6/7 pval = 0.062
1:59:02 AM p 7/8 pval = 0.035
1:59:07 AM p 8/9 pval = 0.019
1:59:13 AM p 9/10 pval = 0.01

Original vs E:\SUMMER TESTS 2005\HQ180\14_Vivaldi RV93 [MPC 1.15v].wav
Playback Range: 01.134 to 04.214
1:59:40 AM p 1/1 pval = 0.5
2:00:01 AM f 1/2 pval = 0.75
2:00:09 AM p 2/3 pval = 0.5
2:00:14 AM p 3/4 pval = 0.312
2:00:24 AM p 4/5 pval = 0.187
2:00:27 AM p 5/6 pval = 0.109
2:00:34 AM p 6/7 pval = 0.062
2:00:38 AM p 7/8 pval = 0.035
2:00:43 AM f 7/9 pval = 0.089
2:00:46 AM p 8/10 pval = 0.054


LAME is worst (problems are some distortions and a lack of precision) followed by faac and then Nero Digital which is –by far- the most efficient encoding¹. aoTuV has something strange: a kind of mix between thinness and fatness. The fatness issue also occurred with sample_04. MPC was transparent: I rated the reference and missed by hair’s breadth the ABX test (pval=0.054).
¹ 168 kbps against 201 kbps (vorbis) and 213 (mpc) which are better, and 193 (faac) and 187 (mp3) which are worse.


Sample 15: Troisème Ballet
Short description: Concerto for bagpipe played on period instruments and including some others rare and rustic instruments (like hurdy-gurdy).
- See
this page for more informations about these old instruments.

Possible problems: ringing for bagpipe, lack of details for hurdy-gurdy.
replaygain_sample_gain: +3.21 dB (indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1L = E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MP3 LAME].wav
2L = E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [AAC faac].wav
3L = E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [vorbis aoTuV].wav
4L = E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [AAC Nero Digital].wav
5L = E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MPC 1.15v].wav

---------------------------------------
General Comments:
---------------------------------------
2L File: E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [AAC faac].wav
2L Rating: 3.0
2L Comment: bagpipe sounds false, with annoying distortions. Slight warbling problems also (faac?).
---------------------------------------
4L File: E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [AAC Nero Digital].wav
4L Rating: 4.5
4L Comment: Distorted.
---------------------------------------
5L File: E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MPC 1.15v].wav
5L Rating: 4.5
5L Comment: Bagpipe is distorted on tonal notes (like an echo within the sound).
---------------------------------------

ABX Results:
Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MPC 1.15v].wav
10 out of 12, pval = 0.019
Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MP3 LAME].wav
10 out of 16, pval = 0.227
Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [AAC Nero Digital].wav
9 out of 12, pval = 0.072
Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [vorbis aoTuV].wav
5 out of 12, pval = 0.806


---- Detailed ABX results ----
Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MPC 1.15v].wav
Playback Range: 00.000 to 04.404
2:31:23 AM p 1/1 pval = 0.5
2:31:29 AM p 2/2 pval = 0.25
2:31:35 AM p 3/3 pval = 0.125
2:31:41 AM p 4/4 pval = 0.062
2:31:53 AM p 5/5 pval = 0.031
2:32:00 AM p 6/6 pval = 0.015
2:32:05 AM p 7/7 pval = 0.0070
2:32:09 AM f 7/8 pval = 0.035
2:32:14 AM p 8/9 pval = 0.019
2:32:21 AM p 9/10 pval = 0.01
2:32:28 AM f 9/11 pval = 0.032
2:32:36 AM p 10/12 pval = 0.019

Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [MP3 LAME].wav
Playback Range: 00.000 to 15.253
2:25:48 AM p 1/1 pval = 0.5
2:25:54 AM p 2/2 pval = 0.25
2:26:00 AM p 3/3 pval = 0.125
2:26:12 AM f 3/4 pval = 0.312
Playback Range: 09.434 to 13.404
2:26:35 AM f 3/5 pval = 0.5
2:26:40 AM p 4/6 pval = 0.343
2:26:44 AM p 5/7 pval = 0.226
2:26:49 AM f 5/8 pval = 0.363
2:26:53 AM p 6/9 pval = 0.253
2:26:58 AM p 7/10 pval = 0.171
2:27:01 AM p 8/11 pval = 0.113
2:27:09 AM f 8/12 pval = 0.193
2:27:12 AM f 8/13 pval = 0.29
2:27:17 AM p 9/14 pval = 0.211
2:27:21 AM f 9/15 pval = 0.303
2:27:25 AM p 10/16 pval = 0.227

Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [AAC Nero Digital].wav
Playback Range: 00.000 to 04.404
2:30:21 AM p 1/1 pval = 0.5
2:30:27 AM f 1/2 pval = 0.75
2:30:33 AM p 2/3 pval = 0.5
2:30:36 AM p 3/4 pval = 0.312
2:30:40 AM p 4/5 pval = 0.187
2:30:43 AM p 5/6 pval = 0.109
2:30:49 AM f 5/7 pval = 0.226
2:30:53 AM p 6/8 pval = 0.144
2:30:57 AM p 7/9 pval = 0.089
2:31:00 AM p 8/10 pval = 0.054
2:31:06 AM f 8/11 pval = 0.113
2:31:10 AM p 9/12 pval = 0.072

Original vs E:\SUMMER TESTS 2005\HQ180\15_Troisème Ballet [vorbis aoTuV].wav
Playback Range: 00.000 to 04.404
2:28:51 AM p 1/1 pval = 0.5
2:28:57 AM p 2/2 pval = 0.25
2:29:04 AM p 3/3 pval = 0.125
2:29:09 AM f 3/4 pval = 0.312
2:29:15 AM p 4/5 pval = 0.187
2:29:20 AM p 5/6 pval = 0.109
2:29:25 AM f 5/7 pval = 0.226
2:29:29 AM f 5/8 pval = 0.363
2:29:34 AM f 5/9 pval = 0.5
2:29:37 AM f 5/10 pval = 0.623
2:29:42 AM f 5/11 pval = 0.725
2:29:45 AM f 5/12 pval = 0.806


Most encoders are able to encode this sample perfectly or very limited problems. Only exception is faac, which distorts the bagpipe and which also adds a slight warbling, typical of this encoder (see sample_03 for similar artefact)¹. Nero Digital and MPC are both² close to perfection to my ears and they only fail (very slightly) on tonal notes of the bagpipe (10/12 for MPC³ and 9/12 for Nero Digital). LAME and aoTuV are both perfect to my ears (10/16 & 5/12).
¹ It might be interesting to link this poor performance to the consummed bitrate: faac used 224 kbps for this short sample (the highest one for faac on 18 samples, and also higher than other competitors for this sample).
² But Nero Digital is by far the most efficient, using 154 kbps instead of 213 kbps for Musepack
³ As for sample_12 I succeed in ABXing MPC 1.15v whereas 1.14 was transparent last year: did my ears become more sensitive or has 1.15 regressed in some areas?


Sample 16: basson
Short description: solo bassoon with light instrumental accompaniment.
Possible problems: micro-attacks on one moment may be softened or distorted.
replaygain_sample_gain: +0.68 dB
(indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1L = E:\SUMMER TESTS 2005\HQ180\16_basson [AAC Nero Digital].wav
2R = E:\SUMMER TESTS 2005\HQ180\16_basson [MPC 1.15v].wav
3L = E:\SUMMER TESTS 2005\HQ180\16_basson [MP3 LAME].wav
4L = E:\SUMMER TESTS 2005\HQ180\16_basson [vorbis aoTuV].wav
5L = E:\SUMMER TESTS 2005\HQ180\16_basson [AAC faac].wav

---------------------------------------
General Comments:
---------------------------------------
1L File: E:\SUMMER TESTS 2005\HQ180\16_basson [AAC Nero Digital].wav
1L Rating: 4.0
1L Comment: Excellent, but audible distortions during 13.0 - 17.0 range.
---------------------------------------
2R File: E:\SUMMER TESTS 2005\HQ180\16_basson [MPC 1.15v].wav
2R Rating: 4.5
2R Comment: Tiny difference during 13.0 - 17.0
---------------------------------------
3L File: E:\SUMMER TESTS 2005\HQ180\16_basson [MP3 LAME].wav
3L Rating: 4.2
3L Comment: Excellent, but again, small distortions during 13.0 - 17.0 (less than 1L and slightly more than 2R)
---------------------------------------
4L File: E:\SUMMER TESTS 2005\HQ180\16_basson [vorbis aoTuV].wav
4L Rating: 3.5
4L Comment: Sound, excellent, becomes "fat" during the problematic 13-17 range.
---------------------------------------
5L File: E:\SUMMER TESTS 2005\HQ180\16_basson [AAC faac].wav
5L Rating: 2.5
5L Comment: The most distorted. Weak performance during 13.04 - 17.04 segment
---------------------------------------

ABX Results:
Original vs E:\SUMMER TESTS 2005\HQ180\16_basson [MP3 LAME].wav
8 out of 10, pval = 0.054
Original vs E:\SUMMER TESTS 2005\HQ180\16_basson [AAC Nero Digital].wav
9 out of 10, pval = 0.01
Original vs E:\SUMMER TESTS 2005\HQ180\16_basson [MPC 1.15v].wav
10 out of 12, pval = 0.019
E:\SUMMER TESTS 2005\HQ180\16_basson [MPC 1.15v].wav vs E:\SUMMER TESTS 2005\HQ180\16_basson [MP3 LAME].wav
11 out of 12, pval = 0.0030


---- Detailed ABX results ----
Original vs E:\SUMMER TESTS 2005\HQ180\16_basson [MP3 LAME].wav
Playback Range: 13.002 to 17.037
2:50:57 AM p 1/1 pval = 0.5
2:51:02 AM p 2/2 pval = 0.25
2:51:08 AM p 3/3 pval = 0.125
2:51:16 AM f 3/4 pval = 0.312
2:51:22 AM p 4/5 pval = 0.187
2:51:32 AM p 5/6 pval = 0.109
2:51:38 AM p 6/7 pval = 0.062
2:51:46 AM p 7/8 pval = 0.035
2:51:55 AM p 8/9 pval = 0.019
2:52:03 AM f 8/10 pval = 0.054

Original vs E:\SUMMER TESTS 2005\HQ180\16_basson [AAC Nero Digital].wav
Playback Range: 13.002 to 17.037
2:48:22 AM p 1/1 pval = 0.5
2:48:26 AM p 2/2 pval = 0.25
2:48:29 AM p 3/3 pval = 0.125
2:48:32 AM p 4/4 pval = 0.062
2:48:36 AM p 5/5 pval = 0.031
2:48:39 AM p 6/6 pval = 0.015
2:48:43 AM p 7/7 pval = 0.0070
2:48:47 AM p 8/8 pval = 0.0030
2:48:51 AM f 8/9 pval = 0.019
2:48:55 AM p 9/10 pval = 0.01

Original vs E:\SUMMER TESTS 2005\HQ180\16_basson [MPC 1.15v].wav
Playback Range: 13.002 to 17.037
2:49:17 AM p 1/1 pval = 0.5
2:49:30 AM p 2/2 pval = 0.25
2:49:35 AM f 2/3 pval = 0.5
2:49:42 AM p 3/4 pval = 0.312
2:49:50 AM f 3/5 pval = 0.5
2:49:57 AM p 4/6 pval = 0.343
2:50:03 AM p 5/7 pval = 0.226
2:50:08 AM p 6/8 pval = 0.144
2:50:15 AM p 7/9 pval = 0.089
2:50:23 AM p 8/10 pval = 0.054
2:50:27 AM p 9/11 pval = 0.032
2:50:34 AM p 10/12 pval = 0.019

E:\SUMMER TESTS 2005\HQ180\16_basson [MPC 1.15v].wav vs E:\SUMMER TESTS 2005\HQ180\16_basson [MP3 LAME].wav
Playback Range: 12.921 to 15.443
2:53:48 AM p 1/1 pval = 0.5
2:53:55 AM p 2/2 pval = 0.25
2:53:59 AM p 3/3 pval = 0.125
2:54:03 AM p 4/4 pval = 0.062
2:54:07 AM p 5/5 pval = 0.031
2:54:11 AM p 6/6 pval = 0.015
2:54:15 AM p 7/7 pval = 0.0070
2:54:19 AM p 8/8 pval = 0.0030
2:54:24 AM f 8/9 pval = 0.019
2:54:28 AM p 9/10 pval = 0.01
2:54:44 AM p 10/11 pval = 0.0050
2:54:48 AM p 11/12 pval = 0.0030


There were no micro-attacks issue (less stronger than those produced by trombone on sample_09. But encoding problems occur elsewhere -and for all encoders- : during one precise range when bassoon is playing at higher volume. It may be interesting to note that all encoders also allocate a lower bitrate than average (150…160 kbps and 175 kbps for MPC).
faac presents the most distorted rendering. The encoder is followed by aoTuV which presents this time (I would say another time… ¹) a fat texture on the described range; this is slightly irritating. The three remaining encoders present non-annoying distortions: Nero Digital, LAME (slightly better) and MPC (even better²).

¹ Same problems on sample_04 and sample_14. Last year, MEGAMIX also obtained an unusually low note on this sample (and exactly the same range).
² ABX comparison opposing directly LAME to MPC: 11/12



Sample 17: Seminarist
Short description: baritone voice accompanied by piano; the sample contains a lot of sibilant consonants.
Possible problems: there’s a big risk of distortion on sibilant.
replaygain_sample_gain: -0.45 dB
(indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1L = E:\SUMMER TESTS 2005\HQ180\17_seminarist [MPC 1.15v].wav
2L = E:\SUMMER TESTS 2005\HQ180\17_seminarist [AAC faac].wav
3R = E:\SUMMER TESTS 2005\HQ180\17_seminarist [vorbis aoTuV].wav
4L = E:\SUMMER TESTS 2005\HQ180\17_seminarist [MP3 LAME].wav
5L = E:\SUMMER TESTS 2005\HQ180\17_seminarist [AAC Nero Digital].wav

---------------------------------------
General Comments:
---------------------------------------
1L File: E:\SUMMER TESTS 2005\HQ180\17_seminarist [MPC 1.15v].wav
1L Rating: 3.2
1L Comment: Sibilant consonent are unatural, "slow".
Issues on voice: during 12.3 - 16.0 the baritone voice has something hard to define: false, hollowed, fluctuating?
---------------------------------------
2L File: E:\SUMMER TESTS 2005\HQ180\17_seminarist [AAC faac].wav
2L Rating: 3.0
2L Comment: Voice is distorted.
---------------------------------------
3R File: E:\SUMMER TESTS 2005\HQ180\17_seminarist [vorbis aoTuV].wav
3R Rating: 4.0
3R Comment: Similar to 1L for sibilants
---------------------------------------
4L File: E:\SUMMER TESTS 2005\HQ180\17_seminarist [MP3 LAME].wav
4L Rating: 3.7
4L Comment: Sibilants are "slow", a bit stretched like chewing-gum. Not shoking (reference file is agressive in my opinion...).
Distorted vocal part between 23.66 - 25.00
---------------------------------------
5L File: E:\SUMMER TESTS 2005\HQ180\17_seminarist [AAC Nero Digital].wav
5L Rating: 3.5
5L Comment: Similar to 1L and 3R for sibilants, but a bit more distorted (clear on 9.00 - 10.00)
---------------------------------------

ABX Results:


faac is not really good and is betrayed by distortions on voice. MPC has presents distortions on sibilant but also on tonal part of voice. Musepack is now used to have problems with voice¹. Nero Digital is slightly better and is followed by LAME; both only show problem on sibilants, moderately annoying (reference is, as I already wrote it during the test, very aggressive). aoTuV is with MPC the one which offers the lesser problem on sibilant, but with a perfect quality on tonal part of the voice. It’s a nice progress since last year².
¹ See comments for sample_03 (chorus), sample_10 (mezzo-soprano) and sample_12 (soprano). Now it’s the turn to male voice to trigger distortion.
² Vorbis MEGAMIX obtained its worst note (2.3) and was clearly worse than LAME 3.97a3; Vorbis aoTuV beta 4 now corrects all problems that were previously audible on voice.



Sample 18: Butterfly Lovers
Short description:violin concerto. Violin is playing alone, then is intertwined with the orchestra
Possible problems: like Liebestod, orchestral strings may be risky for some encoders. Blending between violin and violins is possible.
replaygain_sample_gain: +3.34 dB
(indicative only: files were tested at their original volume)


CODE

ABC/HR for Java, Version 0.5a, 18 août 2005
Testname:

Tester: guruboolez

1L = E:\SUMMER TESTS 2005\HQ180\18_Butterfly [AAC faac].wav
2L = E:\SUMMER TESTS 2005\HQ180\18_Butterfly [AAC Nero Digital].wav
3L = E:\SUMMER TESTS 2005\HQ180\18_Butterfly [MPC 1.15v].wav
4R = E:\SUMMER TESTS 2005\HQ180\18_Butterfly [MP3 LAME].wav
5L = E:\SUMMER TESTS 2005\HQ180\18_Butterfly [vorbis aoTuV].wav

---------------------------------------
General Comments:
---------------------------------------
1L File: E:\SUMMER TESTS 2005\HQ180\18_Butterfly [AAC faac].wav
1L Rating: 4.5
1L Comment: Maybe... I guess... I'm not sure... Go to ABX module.
After ABX: the minor "thing" really exists. Strings are a bit different.
---------------------------------------
2L File: E:\SUMMER TESTS 2005\HQ180\18_Butterfly [AAC Nero Digital].wav
2L Rating: 3.5
2L Comment: Violins are not transparent; there's a form of 'acidity' [I see it like this] on these instruments (I mean orchestra, not solo violin).
---------------------------------------
3L File: E:\SUMMER TESTS 2005\HQ180\18_Butterfly [MPC 1.15v].wav
3L Rating: 3.5
3L Comment: Orchestra sounds fat. It's clear on 8.0 - 12.0 range. I'd bet it's Vorbis and usual coarseness issue.
---------------------------------------
4R File: E:\SUMMER TESTS 2005\HQ180\18_Butterfly [MP3 LAME].wav
4R Rating: 4.8
4R Comment: Minor difference I can 'feel' - I need to ABX them.
After ABX: difference was very hard to catch. Really good encoding, though it's not *fully* transparent.
---------------------------------------

ABX Results:
Original vs E:\SUMMER TESTS 2005\HQ180\18_Butterfly [MP3 LAME].wav
13 out of 16, pval = 0.01
Original vs E:\SUMMER TESTS 2005\HQ180\18_Butterfly [AAC faac].wav
12 out of 12, pval < 0.001
Original vs E:\SUMMER TESTS 2005\HQ180\18_Butterfly [vorbis aoTuV].wav
9 out of 16, pval = 0.401


---- Detailed ABX results ----
Original vs E:\SUMMER TESTS 2005\HQ180\18_Butterfly [MP3 LAME].wav
Playback Range: 09.518 to 16.630
4:09:27 AM p 1/1 pval = 0.5
4:09:33 AM p 2/2 pval = 0.25
4:09:41 AM f 2/3 pval = 0.5
4:09:46 AM p 3/4 pval = 0.312
4:09:54 AM f 3/5 pval = 0.5
4:09:59 AM f 3/6 pval = 0.656
4:10:05 AM p 4/7 pval = 0.5
4:10:10 AM p 5/8 pval = 0.363
4:10:17 AM p 6/9 pval = 0.253
4:10:20 AM p 7/10 pval = 0.171
4:10:34 AM p 8/11 pval = 0.113
4:10:43 AM p 9/12 pval = 0.072
4:10:56 AM p 10/13 pval = 0.046
4:11:02 AM p 11/14 pval = 0.028
4:11:15 AM p 12/15 pval = 0.017
4:11:21 AM p 13/16 pval = 0.01

Original vs E:\SUMMER TESTS 2005\HQ180\18_Butterfly [AAC faac].wav
Playback Range: 03.850 to 13.903
4:06:54 AM p 1/1 pval = 0.5
4:07:06 AM p 2/2 pval = 0.25
4:07:16 AM p 3/3 pval = 0.125
4:07:23 AM p 4/4 pval = 0.062
4:07:30 AM p 5/5 pval = 0.031
4:07:37 AM p 6/6 pval = 0.015
4:07:43 AM p 7/7 pval = 0.0070
4:07:48 AM p 8/8 pval = 0.0030
4:08:01 AM p 9/9 pval = 0.0010
Playback Range: 17.592 to 27.699
4:08:27 AM p 10/10 pval < 0.001
4:08:35 AM p 11/11 pval < 0.001
4:08:40 AM p 12/12 pval < 0.001

Original vs E:\SUMMER TESTS 2005\HQ180\18_Butterfly [vorbis aoTuV].wav
Playback Range: 09.518 to 16.630
4:11:43 AM f 0/1 pval = 1.0
4:11:49 AM f 0/2 pval = 1.0
4:11:56 AM f 0/3 pval = 1.0
4:12:02 AM f 0/4 pval = 1.0
4:12:08 AM p 1/5 pval = 0.968
4:12:11 AM p 2/6 pval = 0.89
4:12:18 AM p 3/7 pval = 0.773
4:12:22 AM p 4/8 pval = 0.636
4:12:30 AM f 4/9 pval = 0.746
4:12:34 AM p 5/10 pval = 0.623
4:12:44 AM p 6/11 pval = 0.5
4:12:48 AM p 7/12 pval = 0.387
4:12:51 AM f 7/13 pval = 0.5
4:13:00 AM p 8/14 pval = 0.395
4:13:08 AM f 8/15 pval = 0.5
4:13:17 AM p 9/16 pval = 0.401


Nero Digital produces an irritating sound. It’s a consequence of encoding problems with violins, already heard and commented for sample_12. MPC offers a similar quality. It sounded fat & coarse, and I was ready to bet that this encoding corresponded to Vorbis, former specialist of this typical problem… faac is in comparison much better. I had nothing but “warm fuzzy feeling” during ABCHR phase, and it was clarified (and verified!) by ABX (12/12): minor distortions on strings. Same thing for LAME: just a feeling on ABCHR which was confirmed by a positive ABX comparison. But the issue is so small that I really can’t describe it nor calling it “distortion”. Nice progress since last year¹. aoTuV is fully transparent to my ears again².
¹ 3.97a3 has a clear ringing on strings, totally solved by alpha 11.
² Again… because Vorbis was also transparent with MEGAMIX last year. It’s interesting to note that it was the only transparent encoding of Vorbis during last listening test. This year, Vorbis aoTuV beta 4 ends with six transparent encodings.

This post has been edited by guruboolez: Aug 21 2005, 19:51
Go to the top of the page
+Quote Post

Posts in this topic
- guruboolez   MPC vs VORBIS vs MP3 vs AAC at 180 kbps   Aug 21 2005, 19:33
- - guruboolez   V. Results and detailed comments Sample 01:...   Aug 21 2005, 19:34
- - guruboolez   Sample 12: Liebestod Short description: Soprano vo...   Aug 21 2005, 19:35
- - guruboolez   VI. Statistical analysis and conclusions It m...   Aug 21 2005, 19:51
|- - Lyx   QUOTE (guruboolez @ Aug 21 2005, 08:51 PM)Vor...   Aug 21 2005, 23:03
- - JeanLuc   Jesus Christ ... now that's one hell of a list...   Aug 21 2005, 19:53
- - guruboolez   Thanks:) Few words to say that I had to split the...   Aug 21 2005, 19:55
- - SirGrey   Wow ! Great work. Very interesting... Especial...   Aug 21 2005, 20:13
- - skelly831   AMAZING Guru! This is the kind of test that m...   Aug 21 2005, 20:25
- - HbG   You're invaluable to this community! And,...   Aug 21 2005, 20:40
- - rjamorim   Awesome! Thank-you very much, Bin Boolez*...   Aug 21 2005, 21:10
- - bug80   I bow, guruboolez. Great test! Now if only m...   Aug 21 2005, 21:19
|- - Yaztromo   Thanks Guru. It's more than obvious you've...   Aug 21 2005, 21:31
- - ff123   Guru, I'm always amazed to read the professio...   Aug 21 2005, 21:37
- - Cyaneyes   Guru.. you've made me very happy I purchased a...   Aug 21 2005, 21:45
- - Lyx   Amazing. When you think he cannot get any better, ...   Aug 21 2005, 21:58
- - CiTay   Well done, guruboolez!   Aug 21 2005, 22:39
- - Busemann   QUOTE • Apple AAC: There's still no VBR mode w...   Aug 21 2005, 22:48
|- - guruboolez   Thanks to you for all support QUOTE (Busemann ...   Aug 22 2005, 02:45
|- - rjamorim   QUOTE (guruboolez @ Aug 21 2005, 10:45 PM)Is ...   Aug 22 2005, 02:59
|- - guruboolez   QUOTE (rjamorim @ Aug 22 2005, 02:59 AM)QUOTE...   Aug 22 2005, 03:09
|- - rjamorim   QUOTE (guruboolez @ Aug 21 2005, 11:09 PM)I...   Aug 22 2005, 03:42
- - NumLOCK   Great work Guruboolez ! It is normal though -...   Aug 21 2005, 23:19
- - vinnie97   holy eff, excellent test, Guru! *not worthy* ...   Aug 21 2005, 23:28
- - kwanbis   excellent guruboolez   Aug 21 2005, 23:47
- - HotshotGG   QUOTE What is the cause of vorbis' noise issue...   Aug 22 2005, 00:09
- - vlada   Great test, thank you. I'm just wondering, how...   Aug 22 2005, 00:19
- - bond   great test! and great work from aoyumi!   Aug 22 2005, 01:34
- - kl33per   Awesome work guru. I don't recall the test one...   Aug 22 2005, 02:08
- - LoFiYo   Great work Mr Guru. Your occasional listening test...   Aug 22 2005, 02:10
- - PabUK   Excellent test, as always. Thanks.   Aug 22 2005, 11:48
- - Sunhillow   Wow! A great test, Guru!   Aug 22 2005, 12:33
- - user   Chapeau Guruboolez ! Congratulations to the p...   Aug 22 2005, 13:02
- - Aoyumi   Thank you for a test and a detailed report, gurubo...   Aug 22 2005, 13:09
|- - arman68   Very good and comprehensive test. Thank you. It c...   Aug 22 2005, 13:25
- - echo   Once again I am impressed by your professional lis...   Aug 22 2005, 13:22
- - Zurman   Impressive ! And quite surprising (vorbis...   Aug 22 2005, 15:20
- - Ivan Dimkovic   Hi, First of all, thanks Guru for your very hard ...   Aug 22 2005, 16:18
|- - Corsair   QUOTE (Ivan Dimkovic @ Aug 22 2005, 04:18 PM)...   Aug 22 2005, 17:16
- - Mr_Rabid_Teddybear   I'm now really happy I recently purchased this...   Aug 22 2005, 16:33
- - HotshotGG   QUOTE Probably, change of ATH, noise/tone masking,...   Aug 22 2005, 17:37
|- - Aoyumi   QUOTE (HotshotGG @ Aug 23 2005, 01:37 AM)QUOT...   Aug 23 2005, 14:43
- - Ivan Dimkovic   QUOTE (Corsair @ Aug 22 2005, 04:16 PM)QUOTE ...   Aug 22 2005, 17:49
- - Digisurfer   Time to play devils advocate. What is the point of...   Aug 22 2005, 21:26
|- - sh1leshk4   QUOTE (Digisurfer @ Aug 23 2005, 03:26 AM)Wha...   Aug 22 2005, 22:05
|- - Lyx   QUOTE (Digisurfer @ Aug 22 2005, 10:26 PM)The...   Aug 22 2005, 22:06
|- - guruboolez   QUOTE (Digisurfer @ Aug 22 2005, 09:26 PM)Tim...   Aug 22 2005, 22:13
|- - rjamorim   QUOTE (Digisurfer @ Aug 22 2005, 05:26 PM)I c...   Aug 22 2005, 22:14
|- - Digisurfer   QUOTE (rjamorim @ Aug 22 2005, 03:14 PM)QUOTE...   Aug 22 2005, 23:20
|- - guruboolez   QUOTE (Digisurfer @ Aug 22 2005, 11:20 PM)Spe...   Aug 22 2005, 23:52
- - germanjulian   thank you for this great test.   Aug 22 2005, 21:35
- - guruboolez   QUOTE (user @ Aug 22 2005, 01:02 PM)Regarding...   Aug 22 2005, 22:02
|- - user   QUOTE (guruboolez @ Aug 22 2005, 10:02 PM)QUO...   Aug 23 2005, 12:04
|- - rjamorim   QUOTE (user @ Aug 23 2005, 08:04 AM)If a (the...   Aug 23 2005, 14:43
||- - guruboolez   QUOTE (rjamorim @ Aug 23 2005, 02:43 PM)QUOTE...   Aug 23 2005, 14:59
||- - rjamorim   QUOTE (guruboolez @ Aug 23 2005, 10:59 AM)Tru...   Aug 23 2005, 15:52
|- - guruboolez   QUOTE (user @ Aug 23 2005, 12:04 PM)The bitra...   Aug 23 2005, 14:49
- - Axon   While I don't doubt that this is a very signif...   Aug 22 2005, 22:08
|- - ching-3   Great work guru, a very good test. I always beli...   Aug 22 2005, 22:13
- - Lyx   I think the flaw in your logic is that you see thi...   Aug 22 2005, 23:33
- - QuantumKnot   It is always very enjoyable and informative to rea...   Aug 23 2005, 00:27
|- - Mo0zOoH   I feel great urge to thank Guruboolez and Aoyumi f...   Sep 2 2005, 00:58
- - vinnie97   I have to agree with Guru. Either these artifacts...   Aug 23 2005, 01:28
- - boiling_ice2k4   nice listening test guruboolez great to see the ...   Aug 23 2005, 03:19
- - HotshotGG   QUOTE What can I say? I am extremely happy with Og...   Aug 23 2005, 05:25
- - DARcode   Awesome piece of work and information, simply awes...   Aug 30 2005, 15:27
- - guruboolez   Here is the bitrate table (based on 150 full track...   Aug 31 2005, 11:19
- - esa372   Wow! Thank you, guruboolez!   Aug 31 2005, 15:26
- - Pio2001   Thank you for the test. I hope it will encourage o...   Sep 1 2005, 22:34
|- - bug80   QUOTE (Pio2001 @ Sep 1 2005, 11:34 PM)And I t...   Sep 1 2005, 22:50
|- - ff123   QUOTE (bug80 @ Sep 1 2005, 01:50 PM)QUOTE (Pi...   Sep 2 2005, 21:37
- - alter4   And in my opinion, the handling pre-echo mechanism...   Sep 2 2005, 06:14
|- - Yaztromo   QUOTE (alter4 @ Sep 2 2005, 06:14 AM)And in m...   Sep 2 2005, 10:34
- - Aoyumi   QUOTE And in my opinion, the handling pre-echo mec...   Sep 3 2005, 04:22
- - alter4   I tested castanets2 with beta4 Now I have no ABX l...   Sep 5 2005, 08:17
|- - Yaztromo   I will test with beta4 in the next couple of days.   Sep 5 2005, 10:02
- - alter4   the first click in castanets2, no comment!   Sep 8 2005, 09:52
|- - Aoyumi   QUOTE (alter4 @ Sep 8 2005, 05:52 PM)the firs...   Sep 8 2005, 14:04
- - alter4   My friends (I ask theirs to help) say that in cast...   Sep 8 2005, 15:23
- - Aoyumi   >alter4 Can the portion which you ABX(ed) be sh...   Sep 11 2005, 07:45
- - liekloo   I know I'm late. Very late even. But I don...   Sep 15 2005, 22:35
|- - zima   Very impressive...I wish I had sometimes this kind...   Sep 16 2005, 19:35
|- - beto   QUOTE (liekloo @ Sep 15 2005, 06:35 PM)I know...   Sep 16 2005, 21:02
- - Seymour   Guruboolez Great test! Thanks. Surely I'l...   Sep 17 2005, 14:39
- - VEG   Wow! Vorbis bested that MPC on 180kbps?! I...   Sep 22 2005, 21:09
|- - kornchild2002   Thanks for the testing. I can't help but thin...   Sep 22 2005, 22:40
|- - stephanV   QUOTE (kornchild2002 @ Sep 22 2005, 11:40 PM)...   Sep 23 2005, 08:42
- - Donunus   I was just wondering if upping lame 3.97 to preset...   Sep 23 2005, 11:37
- - beto   probably yes. but that's an assumption.   Sep 23 2005, 12:23
- - SCIF   Does anybody with good headphones want to abx-test...   Oct 28 2005, 05:47
|- - de Mon   QUOTE (SCIF @ Oct 27 2005, 08:47 PM)Does anyb...   Oct 28 2005, 08:18
|- - SCIF   QUOTE (de Mon @ Oct 28 2005, 05:18 PM)I bet G...   Nov 2 2005, 06:31
- - kuniklo   I just came across this thread today. Thanks to G...   Oct 31 2005, 22:39
- - lextune   Thanks for this interesting and informative post. ...   Jan 6 2006, 21:37
- - Garf   Re-opening so discussion can continue.   Jan 9 2006, 12:13
- - vinnie97   I saw this thread was locked yesterday with no exp...   Jan 9 2006, 23:38
|- - kode54   QUOTE (vinnie97 @ Jan 9 2006, 02:38 PM)I saw ...   Jan 9 2006, 23:49
- - vinnie97   No need to be testy. I didn't see that *other...   Jan 10 2006, 00:30


Closed TopicStart new topic
5 User(s) are reading this topic (5 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 13th July 2014 - 05:13