IPB

Welcome Guest ( Log In | Register )

12 Pages V  « < 5 6 7 8 9 > »   
Reply to this topicStart new topic
AAC at 128kbps v2 listening test - FINISHED
rjamorim
post Mar 3 2004, 01:00
Post #151


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Anyway, AAC quality sucks, according to Microsoft, so we should as well stop discussing this shitty format and move on to WMA standard biggrin.gif

http://www.macnn.com/news/19380


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
JohnV
post Mar 3 2004, 01:10
Post #152





Group: Developer
Posts: 2797
Joined: 22-September 01
Member No.: 6



QUOTE (rjamorim @ Mar 3 2004, 01:57 AM)
QUOTE (JohnV @ Mar 2 2004, 08:51 PM)
Not anykind of reliable approximation of cbr quality from vbr. The coding methods are so different. CBR has often different kind of issues than VBR, and CBR can avoid undercoding problems which may happen with VBR especially at mid-low bitrates like near 128kbps average. The tweaking of CBR and VBR profiles must be done totally separately.

We are not trying to approximate CBR from VBR. We are approximating a theoretical VBR mode that stayed at an average bitrate of 128kbps across test samples.

It's pointless, because it's not tweaked and emphasis in real life can be put on psychoacoustic settings which lower bitrate but decrease the quality relatively less than some other setting or this so called linearly approximated theoretical VBR-mode preset.


--------------------
Juha Laaksonheimo
Go to the top of the page
+Quote Post
ScorLibran
post Mar 3 2004, 01:17
Post #153





Group: Banned
Posts: 769
Joined: 1-July 03
Member No.: 7495



QUOTE (Ivan Dimkovic @ Mar 2 2004, 06:49 PM)
QUOTE
ScorLibran
1. Tell me how it's wrong. That's been addressed to some extent (though I'm not enough of an expert to know what the answer really means)... and ...


Psychoacoustic coders do not scale quality with bit rate in a linear fashion - meaning that you can't just project quality result to some other bit rate by simple proportion.

Understood. So naturally my next question would be...

Then what does the math look like for at least getting a fairly accurate composite rating? Or at least closer than a linear calculation?

QUOTE (Ivan Dimkovic @ Mar 2 2004, 06:49 PM)
QUOTE

From among the Eternal Questions of Psychoacoustic Audio Encoding™: "My quality target is X. How many files encoded to my quality target with Nero AAC can I put on my 20GB hard drive? OK, how many files encoded to the same quality target with QT AAC can I put in the same space?"


Well - we did these tests before listening test, and Nero ended up in 130.5 (correct me if I am wrong) in average - which indicates how much stuf you would be able store.

Of course, it depends on signal statistics - so if you want to know exactly how much, you'll either have to encode files, or to use CBR. Different music has different masking patterns, and if you use 'real' VBR - it is not easy to predict final bit rate without, at least, psychoacoustic processing.

So to answer the question of the non-expert music encoder asking how to calculate capacity requirements for each encoding format...

"Encode your collection with a setting you know you like, and if you can't fit all the music you want to onto your drive, then try a little lower setting and start over."

-- Not feasible.

"Use CBR."

-- Not good advice for the best sound quality, in many cases.

So, that person is stuck without a good answer, unless some kind of composite rating system is used with test results such as these.

First, I'd need to have an example of math that would be more accurate than my linear calculations.

And regarding variances across samples and types of music, well, that's where the (now mentioned at least 3 times) qualification of composite rating interpretation would come into play. Everyone knew going in that in Roberto's test there could only be a limited number of samples, and it was my understanding that it would be safe to quote those results with the qualifier of "with the samples tested, here's how the codecs performed..."

Once we have a more accurate calculation method, we should just go with what we have so far, and as more tests are done, use those results to provide new composite ratings over time.

I'm just trying to speak for the hundreds of people that have posed these questions to me over the past year, and never have gotten a good, straightforward answer. And if my presentation of these questions and ideas is getting a bit too "simplistic", then forgive me, but inquiring minds want to know! wink.gif

This post has been edited by ScorLibran: Mar 3 2004, 01:18
Go to the top of the page
+Quote Post
QuantumKnot
post Mar 3 2004, 01:22
Post #154





Group: Developer
Posts: 1245
Joined: 16-December 02
From: Australia
Member No.: 4097



QUOTE (rjamorim @ Mar 3 2004, 10:00 AM)
Anyway, AAC quality sucks, according to Microsoft, so we should as well stop discussing this shitty format and move on to WMA standard biggrin.gif

http://www.macnn.com/news/19380

hehehe Interesting article.

QUOTE
It’s true: Apple’s AAC cuts sound great with the tiny little speakers that come with computers. And they sound pretty good on an original (but AAC upgraded) iPod through the stock headphones. But listen through good headphones and what you’ll hear is dull-sounding bass, slightly sibilant voice quality and a lack of three-dimensionality.


Three-dimensionality? Perhaps he lacked those special blue and red headphones laugh.gif
Go to the top of the page
+Quote Post
tcristy
post Mar 3 2004, 01:34
Post #155





Group: Members
Posts: 7
Joined: 3-March 04
Member No.: 12423



I encoded 145 rock songs ranging from classic rock through metal with Compaact Q5 VBR and no low-pass filtering (I hate the low pass, which is why I use Compaact and not iTunes, which sounds dead to me) and got the following distribution:

http://home.columbus.rr.com/tcristy/CPDistro.jpg

The average was 126 kpbs.

The highest bit rate song at a bit over 143 kbps average was ZZ Top's Sleeping Bag, which has a synthed cymbal at about 6 beats per second through the whole thing.

Tim

This post has been edited by tcristy: Mar 3 2004, 01:38
Go to the top of the page
+Quote Post
bidz
post Mar 3 2004, 01:49
Post #156





Group: Members
Posts: 351
Joined: 27-December 02
From: Norway
Member No.: 4258



If all encoders was tested in CBR mode, this discussion wouldn't be a issue wink.gif

That would - atleast - be a 100% fair comparison.


--------------------
myspace.com/borgei - last.fm/user/borgei
Go to the top of the page
+Quote Post
guruboolez
post Mar 3 2004, 02:25
Post #157





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Last year, Roberto was criticized for choosing CBR for all encoders, especially for Nero AAC, which had a VBR mode. Some people were sure that Nero AAC with VBR would beat QuickTime AAC CBR... That was another discussion, and people didn't considered the forced CBR comparison as something fair. And that's probabably true.
Go to the top of the page
+Quote Post
kwanbis
post Mar 3 2004, 02:51
Post #158





Group: Developer (Donating)
Posts: 2362
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (bidz @ Mar 3 2004, 12:49 AM)
That would - atleast - be a 100% fair comparison.

it won't, we are trying to compare what is the "standard" among them, and the way it was done is perfect, for me at least.


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
bidz
post Mar 3 2004, 02:59
Post #159





Group: Members
Posts: 351
Joined: 27-December 02
From: Norway
Member No.: 4258



QUOTE (kwanbis @ Mar 2 2004, 05:51 PM)
QUOTE (bidz @ Mar 3 2004, 12:49 AM)
That would - atleast - be a 100% fair comparison.

it won't, we are trying to compare what is the "standard" among them, and the way it was done is perfect, for me at least.

Well, IMHO, a perfect comparison consists of identical bitrates. If one encoder uses VBR, and gets 140kbps - thats not fair to compare against a 128kbps CBR codec IMHO. (even though if the CBR wins).

Just my opinion.


--------------------
myspace.com/borgei - last.fm/user/borgei
Go to the top of the page
+Quote Post
ScorLibran
post Mar 3 2004, 03:31
Post #160





Group: Banned
Posts: 769
Joined: 1-July 03
Member No.: 7495



For my own concern, I'm not so interested in what's "fair". The bitrate a codec averages on a sample is what it averages...whether it's high or low is incidental to me.

What I'd like to see, though, is some kind of "quality per bit" rating, or a "composite rating". Like I said, although VBR doesn't really put relevance on bitrate, people do, since bitrate determines filesize, and filesize is used to determine how much music you can fit into a limited space.

Simple concept, but it seems the answer may be quite complex. tongue.gif
Go to the top of the page
+Quote Post
rjamorim
post Mar 3 2004, 04:29
Post #161


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



The bitrate distribution table is up
http://www.rjamorim.com/test/aac128v2/results.html

Just under the individual plots, as usual.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
bidz
post Mar 3 2004, 04:45
Post #162





Group: Members
Posts: 351
Joined: 27-December 02
From: Norway
Member No.: 4258



140 average against 128 .. (Nero - iTunes). I'm really curious on how Nero's rating would be if it would use 128kbps CBR... Oh well..


--------------------
myspace.com/borgei - last.fm/user/borgei
Go to the top of the page
+Quote Post
Stux
post Mar 3 2004, 06:30
Post #163


3ivx developer


Group: Developer
Posts: 52
Joined: 10-December 03
Member No.: 10346



QUOTE (Ivan Dimkovic @ Mar 3 2004, 10:07 AM)
QUOTE
So that warrants allowing a codec to use more bits than allowed by the test setup?


Hmm.. we had this discussion before - and if I remember correctly, in the time of 128 extension test it was decided to use values for near 128 kb/s on average content, not on test items.

The fact that encoder used more bits on this particular sample set just means that it judged them as "hard to encode"

Of course, you could fine-scale encoder for each sample to even give you 128 kb/s for each sample with VBR - but what's the use of that - unless encoder has 2-pass VBR supported and used by most users?

QUOTE
What are you trying to say there?


That linear scaling of quality to projected bit rate is dead flat wrong.

By telling a CBR encoder to encode at 128kbps you are telling it to sacrifice quality to encode at 128kbps.

Then you place VBR encoders in the same situation and tell them to sacrifice bitrate accuracy to maintain quality... in a quality oriented test this is not fair.

Personally (and we do this in video tests), if a codec operates best in VBR mode then a VBR setting should be used which comes as close as possible to the CBR bitrate ON THAT SAMPLE, otherwise what's the justification for not using the VBR bitrate with the CBR encoder, thus asking it to constrain its quality to the same bitrate as the VBR encoder.

Unless of course the granularity on VBR aac encoders isn't that crash hot.


Finding the VBR settings which matches a CBR bitrate might take multiple encodings, but I think its the only way to have a fair test, the test is not how each encoder does at "ultra streaming" quality, but rather at 128kbps.


--------------------
http://www.3ivx.com
Go to the top of the page
+Quote Post
sven_Bent
post Mar 3 2004, 06:52
Post #164





Group: Members
Posts: 691
Joined: 15-December 01
From: Denmark
Member No.: 655



QUOTE (rjamorim @ Mar 3 2004, 01:00 AM)
Anyway, AAC quality sucks, according to Microsoft, so we should as well stop discussing this shitty format and move on to WMA standard biggrin.gif

http://www.macnn.com/news/19380

This article is a joke

He compared the downloaded track to HIS original CD.... who say its from the same source ?

anyway her clearly take time to say bad things about AAC but he really lacks to mention any better alternative.

The burned disc did NOT play in any of my CD players. Not in the ones hooked up to my stereo, my portable players, or even in an old laptop without DVD capabilities. Nor did they play on either of my older MP3 players.

(May 15 update: It turned out the blank CD I was using was bad. I used other discs and was able to play them on my CD players.)


So it took him NINE days to figure out the CD was bad..... now thats a technical guy i can put my trust in...DOH
He is clearly missing the science rule number one: "anything has to be done twice"


<i>Any device that can play a DVD can play burned copies of Apple’s AAC-compressed songs</i>
What the HECK. this is just plain wrong.. unless he burns an AudioCD which he DOES NOT mention at all.

This post has been edited by sven_Bent: Mar 3 2004, 07:15


--------------------
Sven Bent - Denmark
Go to the top of the page
+Quote Post
Jerethi
post Mar 3 2004, 07:31
Post #165





Group: Members
Posts: 69
Joined: 12-January 02
Member No.: 996



QUOTE (rjamorim @ Mar 2 2004, 04:00 PM)
Anyway, AAC quality sucks, according to Microsoft, so we should as well stop discussing this shitty format and move on to WMA standard biggrin.gif

http://www.macnn.com/news/19380

This article is a pretty pathetic attempt at journalism. He has obviously done little or no research, and clearly evinces a biased point of view. This article is completely devoid of merit.

Either he's accusing Dolby of falsely representing the fact that AAC has undergone extensive listening tests, or he's claiming to have the shiniest golden ears this world has ever known.

What the hell are we paying editors for these days? dry.gif

EDIT: Well, I guess there's some meaningful content in the comments he makes about iTunes online store, but this hardly what I would call a "review."

This post has been edited by Jerethi: Mar 3 2004, 07:33


--------------------
http://www.assortmentstoday.com
Go to the top of the page
+Quote Post
sony666
post Mar 3 2004, 08:00
Post #166





Group: Members
Posts: 573
Joined: 22-February 02
Member No.: 1375



I would have preferred only CBR encodings in this test and the previous mp3 one. but that's just me smile.gif
hope there will be a poll regarding this question for the next test.
Go to the top of the page
+Quote Post
rjamorim
post Mar 3 2004, 08:16
Post #167


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (sony666 @ Mar 3 2004, 04:00 AM)
I would have preferred only CBR encodings in this test and the previous mp3 one. but that's just me smile.gif
hope there will be a poll regarding this question for the next test.

Impossible. How do you expect me to come up with CBR Musepack? Or VBR iTunes?

Also, Vorbis zealots would slaughter me if I forced it to use CBR.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Continuum
post Mar 3 2004, 08:24
Post #168





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (rjamorim @ Mar 3 2004, 04:29 AM)
The bitrate distribution table is up
http://www.rjamorim.com/test/aac128v2/results.html

Great!

Can we get the decryption key now too? biggrin.gif
Go to the top of the page
+Quote Post
rjamorim
post Mar 3 2004, 08:28
Post #169


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



OMG! yes.

http://www.rjamorim.com/test/aac128v2/comments/aac128v2.key


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
harashin
post Mar 3 2004, 08:43
Post #170





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



Decrypted my results. Somehow I cannot find my result for sample05 at http://www.rjamorim.com/test/aac128v2/comments/results05/.
Other results are available there.

CODE
ABC/HR for Java Version 0.4b3 SE, 20 2月 2004
Testname: Hongroise listening test

Tester: harashin

1L = Sample05\Hongroise_1.wav
2R = Sample05\Hongroise_5.wav
3L = Sample05\Hongroise_2.wav
4R = Sample05\Hongroise_3.wav
5R = Sample05\Hongroise_4.wav

---------------------------------------
General Comments:
---------------------------------------
2R File: Sample05\Hongroise_5.wav
2R Rating: 4.5
2R Comment:
---------------------------------------

ABX Results:
Original vs Sample05\Hongroise_1.wav
   25 out of 41, pval = 0.105
Original vs Sample05\Hongroise_2.wav
   15 out of 28, pval = 0.425
Original vs Sample05\Hongroise_3.wav
   16 out of 32, pval = 0.569
Original vs Sample05\Hongroise_4.wav
   6 out of 14, pval = 0.788
Original vs Sample05\Hongroise_5.wav
   6 out of 6, pval = 0.015


This post has been edited by harashin: Mar 3 2004, 08:47


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
sony666
post Mar 3 2004, 08:46
Post #171





Group: Members
Posts: 573
Joined: 22-February 02
Member No.: 1375



I subconsciously associated multiformat with "mp3 vs. AAC" by wishful thinking, and unjustly pushed aside mpc and vorbis in my (sleepy) mind. my apologies. crying.gif

This post has been edited by sony666: Mar 3 2004, 08:47
Go to the top of the page
+Quote Post
rjamorim
post Mar 3 2004, 08:47
Post #172


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



For some strange reason, I couldn't decrypt some of the Sample05 (and a few other samples) results. sad.gif

It must be a bug in ABC/HR Java

Here's an example:
http://pessoal.onda.com.br/rjamorim/results09.zip

BTW: I already reported the bug to schnofler.

This post has been edited by rjamorim: Mar 3 2004, 08:49


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
harashin
post Mar 3 2004, 08:53
Post #173





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



QUOTE (rjamorim @ Mar 3 2004, 04:47 PM)
For some strange reason, I couldn't decrypt some of the Sample05 (and a few other samples) results. sad.gif

It must be a bug in ABC/HR Java

Here's an example:
http://pessoal.onda.com.br/rjamorim/results09.zip

BTW: I already reported the bug to schnofler.

I see. The results09.erf can't be decrypted here, either.


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
Continuum
post Mar 3 2004, 10:02
Post #174





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (harashin @ Mar 3 2004, 08:43 AM)
Decrypted my results. Somehow I cannot find my result for sample05 at http://www.rjamorim.com/test/aac128v2/comments/results05/.
Other results are available there.

My result for sample 5 is missing there as well, for some unknown reason. No problem decrypting it locally:

CODE
ABC/HR for Java Version 0.4b3 SE, 20 Februar 2004
Testname: Hongroise listening test

Tester: Continuum

1L = Sample05\Hongroise_3.wav
2R = Sample05\Hongroise_2.wav
3L = Sample05\Hongroise_5.wav
4R = Sample05\Hongroise_4.wav
5L = Sample05\Hongroise_1.wav

---------------------------------------
General Comments:
---------------------------------------
1L File: Sample05\Hongroise_3.wav
1L Rating: 3.0
1L Comment: pre-echo? smeared at 10.6
---------------------------------------
2R File: Sample05\Hongroise_2.wav
2R Rating: 4.5
2R Comment: ABC-HR click moved (start pos 9.03)
little problem at sec 15
---------------------------------------
3L File: Sample05\Hongroise_5.wav
3L Rating: 2.0
3L Comment: severly smeared at ~10.6
---------------------------------------
4R File: Sample05\Hongroise_4.wav
4R Rating: 4.8
4R Comment: very little pre-echo at sec 22-23
---------------------------------------
5L File: Sample05\Hongroise_1.wav
5L Rating: 2.5
5L Comment: sec 15
---------------------------------------

ABX Results:
Original vs Sample05\Hongroise_2.wav
   1 out of 1, pval = 0.5
Original vs Sample05\Hongroise_1.wav
   5 out of 5, pval = 0.031


I find this sample particulary interesting. All codecs had problems at different sections. And looking at the bitrate table,
CODE
           Nero  Real  Faac  Comp!  iTunes
Hongroise   148   128   105    123    128

I am quite shocked by Nero's performance. Faac on the other hand got a respectable rating, considering its low bitrate.
Go to the top of the page
+Quote Post
spoon
post Mar 3 2004, 10:27
Post #175


dBpowerAMP developer


Group: Developer (Donating)
Posts: 2745
Joined: 24-March 02
Member No.: 1615



QUOTE (rjamorim @ Mar 2 2004, 11:23 PM)
Nope. Faac and Compaact! are VBR, the settings I used output average 128kbps over a very big amount of tracks, and still they behaved, staying at 128kbps +-4

I need to publish the bitrate deviation table ASAP >_<

The audio tracks I compresses were a mix of Pop, Rock, Rap - popular tracks. I could upload a playlist if required.

If you were to publish the bitrates over 2GB then any squabbling should stop, anyone looking at the results page for the first time could get the wrong impression that Nero was encoded too high.


--------------------
Spoon http://www.dbpoweramp.com
Go to the top of the page
+Quote Post

12 Pages V  « < 5 6 7 8 9 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 28th August 2014 - 13:10