IPB

Welcome Guest ( Log In | Register )

11 Pages V   1 2 3 > »   
Reply to this topicStart new topic
Multiformat@128kbps listening test - FINISHED
rjamorim
post May 24 2004, 06:33
Post #1


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello.

I'd like to announce the results of the Multiformat at 128kbps listening test

Vorbis aoTuV is tied to Musepack at first place, Lame MP3 is tied to iTunes AAC at second place, WMA Standard is in third place and Atrac3 gets last place.

The results page is here:
http://www.rjamorim.com/test/multiformat128/results.html

For those in a hurry, here are the zoomed overall results:


Big thanks to everyone that helped and participated.

Best regards;
Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
magic75
post May 24 2004, 06:45
Post #2





Group: Members
Posts: 511
Joined: 2-December 02
Member No.: 3959



Now that was a surprise... Lame as good as AAC??? Anyone expected that?
Go to the top of the page
+Quote Post
ScorLibran
post May 24 2004, 06:55
Post #3





Group: Banned
Posts: 769
Joined: 1-July 03
Member No.: 7495



Vorbis (aoTuV) and MPC tied for first place. LAME and iTunes tied for second. Then WMA-S in third, and ATRAC3 at the back of the pack.

Funny that there was no real consistency this time across music types with the formats tested. Tends to oppose theories about certain formats excelling with certain types of music. At least among these samples.

This post has been edited by ScorLibran: May 24 2004, 06:55
Go to the top of the page
+Quote Post
bidz
post May 24 2004, 07:01
Post #4





Group: Members
Posts: 351
Joined: 27-December 02
From: Norway
Member No.: 4258



What the! huh.gif ... surprised!


--------------------
myspace.com/borgei - last.fm/user/borgei
Go to the top of the page
+Quote Post
QuantumKnot
post May 24 2004, 07:01
Post #5





Group: Developer
Posts: 1245
Joined: 16-December 02
From: Australia
Member No.: 4097



Whoa, look at aoTuV!! ohmy.gif

It is now as good as MPC. Very good work, Aoyumi. Vorbis is now back in the spotlight. biggrin.gif
Go to the top of the page
+Quote Post
harashin
post May 24 2004, 07:07
Post #6





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



I believed Musepack would win the test especially such bitrate range(-q4.15). Anyway, it's very interesting result, good job Roberto and all participants.


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
guruboolez
post May 24 2004, 07:07
Post #7





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Surprisingly, MPC 1.14 (same tested last year) isn't tied anymore with iTunes AAC, but “win”.
ATRAC3 (minidisc) is obviously a poor encoding solution.
aoTuV is without doubt a great step behind for Vorbis!
Go to the top of the page
+Quote Post
rjamorim
post May 24 2004, 07:15
Post #8


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



The codes:

1 - Vorbis aoTuV
2 - Musepack
3 - Lame MP3
4 - iTunes AAC
5 - Atrac3
6 - WMA Std.

The decryption key:
http://www.rjamorim.com/test/multiformat12...multiformat.key


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
JohnV
post May 24 2004, 07:18
Post #9





Group: Developer
Posts: 2797
Joined: 22-September 01
Member No.: 6



Very good results by aoTuV. It seems all the others have a new target for 128kbps quality now.
One thing which this test shows is that VBR coding (aoTuV, MPC) is definitely way to go for 128kbps, and with good enough VBR tweaking it's certainly possible to be clearly better than CBR (iTunes).


--------------------
Juha Laaksonheimo
Go to the top of the page
+Quote Post
rjamorim
post May 24 2004, 07:21
Post #10


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (JohnV @ May 24 2004, 03:18 AM)
One thing which this test shows is that VBR coding (AoTuV, MPC) is definitely way to go for 128kbps, and with good enough VBR tweaking it's certainly possible to be clearly better than CBR (iTunes).

Yes. That is also true for Lame. With a very good VBR implementation, it got close to the best AAC implementation at that bitrate.

Let's hope Apple implements VBR in their codec, and Ahead improves their implementation considerably.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ScorLibran
post May 24 2004, 07:22
Post #11





Group: Banned
Posts: 769
Joined: 1-July 03
Member No.: 7495



QUOTE (harashin @ May 24 2004, 01:07 AM)
I believed Musepack would win the test especially such bitrate range(-q4.15).

I thought so too.

I anticipated a tie between MPC and QT-AAC, then Vorbis in second place, then LAME, then WMA-S and ATRAC at the back. Vorbis and QT-AAC both surprised me.
Go to the top of the page
+Quote Post
harashin
post May 24 2004, 07:29
Post #12





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



My browsers(Firefox, MSIE) don't show test comments correctly. Also, the title of this page seems to be wrong.


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
rjamorim
post May 24 2004, 07:33
Post #13


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (harashin @ May 24 2004, 03:29 AM)
My browsers(Firefox, MSIE) don't show test comments correctly.

It's XML. IE should show something like this:
http://esc17.midphase.com/~calmerc/screenshots/screen-1.jpg

XML is worse for readability but easier to be parsed. That's why Schnofler switched to XML results in recent versions of ABC/HR Java.

QUOTE
Also, the title of this page seems to be wrong.


Fixed. Thanks for reporting.

This post has been edited by rjamorim: May 24 2004, 07:35


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
harashin
post May 24 2004, 07:38
Post #14





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



QUOTE (rjamorim @ May 24 2004, 03:33 PM)
XML is worse for readability but easier to be parsed. That's why Schnofler switched to XML results in recent versions of ABC/HR Java.

I expected something like in raw *.txt format. Thanks for clarification.


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
rjamorim
post May 24 2004, 07:40
Post #15


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (harashin @ May 24 2004, 03:38 AM)
I expected something like in raw *.txt format. Thanks for clarification.

Schnofler already has a converter from xml -> txt in ABC/HR. But it only works for encrypted results ATM. Hopefully he'll add support for already decrypted results.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Bonzi
post May 24 2004, 08:23
Post #16


A/V Moderator


Group: Members
Posts: 278
Joined: 22-February 03
Member No.: 5132



Wow, what really impresses me is that I don't think there was one sample where the vorbis encoder did poorly. This is a little shocking after last test. Excellent work aoTuV!
Go to the top of the page
+Quote Post
Der_Iltis
post May 24 2004, 08:32
Post #17





Group: Members
Posts: 101
Joined: 9-August 03
Member No.: 8270



Surprise surprise!
I hope this'll give vorbis development a new boost.
Go to the top of the page
+Quote Post
Gabriel
post May 24 2004, 08:34
Post #18


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Oh! Joy!
Go to the top of the page
+Quote Post
rjamorim
post May 24 2004, 08:38
Post #19


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Gabriel @ May 24 2004, 04:34 AM)
Oh! Joy!

laugh.gif

I'm happy my test is spreading happiness.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
bond
post May 24 2004, 08:50
Post #20





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



woow, now thats what i not expected

- vorbis aotuv: vorbis is back, and i am proud to have helped finding out what vorbis encoder should be used smile.gif
- mpc vs aac: funny that mpc was that better than itunes (with a only 0.15 higher setting than in the last test)
- wma9: lol, worse than mp3! (and i even wonder that it got rated that high, even at 128 it had this metallic sound sometimes) -> go away m$
- atrac3: even worse than wma9 -> go away sony

and if you take this test as a comparison between some online music stores (itunes vs. wma9 based ones vs. sonys new store) itunes clearly comes out as the winner, leaving wma9 behind by far!


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
FireStarter
post May 24 2004, 08:56
Post #21





Group: Banned
Posts: 105
Joined: 17-April 04
Member No.: 13570



I see there is a very small margin between mpc and aoTuv, how would aoTuv react
in higher bitrates.?
Go to the top of the page
+Quote Post
JeanLuc
post May 24 2004, 08:59
Post #22





Group: Members
Posts: 1311
Joined: 4-June 02
From: Cologne, Germany
Member No.: 2213



Very interesting results ...

I think it could be an interesting addition to show the bitrate for each encoder in the specific diagrams for each sample ...


--------------------
The name was Plex The Ripper, not Jack The Ripper
Go to the top of the page
+Quote Post
dev0
post May 24 2004, 09:00
Post #23





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



The more I think of it the more impressed I am with the performance of LAME. Very good work Gabriel (and consider changing -V 5 default --athaa-sensitivity).


--------------------
"To understand me, you'll have to swallow a world." Or maybe your words.
Go to the top of the page
+Quote Post
Raptus
post May 24 2004, 09:00
Post #24





Group: Members
Posts: 73
Joined: 22-February 04
From: Germany
Member No.: 12191



How many results were discarded because of ranked refs?
Go to the top of the page
+Quote Post
rjamorim
post May 24 2004, 09:06
Post #25


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Raptus @ May 24 2004, 05:00 AM)
How many results were discarded because of ranked refs?

54

Mind you that I didn't discard results that ranked the reference but on that sample pair ABXd the samples to a pval of 0.05 or less.

This post has been edited by rjamorim: May 24 2004, 09:08


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post

11 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 29th December 2014 - 16:33