IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Download.com does an internal listening test
spoon
post Jun 5 2005, 20:53
Post #1


dBpowerAMP developer


Group: Developer (Donating)
Posts: 2758
Joined: 24-March 02
Member No.: 1615



Seems to be a blind listening test done with high end audoi gear in an audio studio:

http://www.download.com/Audio-Codec-Shoot-...20-5146271.html

Results that Microsoft might be very happy with.


--------------------
Spoon http://www.dbpoweramp.com
Go to the top of the page
+Quote Post
sTisTi
post Jun 5 2005, 21:09
Post #2





Group: Members
Posts: 385
Joined: 25-June 04
Member No.: 14895



QUOTE (spoon @ Jun 5 2005, 11:53 AM)
Seems to be a blind listening test done with high end audoi gear in an audio studio:

http://www.download.com/Audio-Codec-Shoot-...20-5146271.html

Results that Microsoft might be very happy with.
*

The fact that 128 encodes were very often considered to better than 192 ones makes me very sceptical of this test. WMA 128 better than Ogg 192? Come on... tongue.gif I am not especially sensitive concerning compression artefacts, but the metallic ringing of a typical WMA@128 really hurts my ears...

This post has been edited by sTisTi: Jun 5 2005, 21:13


--------------------
Proverb for Paranoids: "If they can get you asking the wrong questions, they don't have to worry about answers."
-T. Pynchon (Gravity's Rainbow)
Go to the top of the page
+Quote Post
iehova
post Jun 5 2005, 21:13
Post #3





Group: Members
Posts: 81
Joined: 31-July 04
Member No.: 15910



they should also have used LAME instead of iTunes to encode their MP3s

...and of course an up to date version of Vorbis, instead of "Xiph.Org libVorbis I 20030909" ^^

This post has been edited by iehova: Jun 5 2005, 21:14


--------------------
Friends don't let friends use lossy codecs. (char0n)
Go to the top of the page
+Quote Post
Busemann
post Jun 5 2005, 22:22
Post #4





Group: Members
Posts: 730
Joined: 5-January 04
Member No.: 10970



That the higher bitrate clips do worse seems to indicate that this is a test to find the codec "which sounds best to you". Not most true to the original.

Bring in another crowd an the results could be the complete opposite.
Go to the top of the page
+Quote Post
HotshotGG
post Jun 5 2005, 23:16
Post #5





Group: Members
Posts: 1593
Joined: 24-March 02
From: Revere, MA
Member No.: 1607



QUOTE
The fact that 128 encodes were very often considered to better than 192 ones makes me very sceptical of this test. WMA 128 better than Ogg 192? Come on... tongue.gif I am not especially sensitive concerning compression artefacts, but the metallic ringing of a typical WMA@128 really hurts my ears.


This is the stuff that pisses me off right here. I say we all petition and send them a nasty e-mail or something pathetic wink.gif. Who did they hire a Johnny Nothing off the street to do this? It says the test is "non-scientific" how would you expect anyone to take you seriously?

This post has been edited by HotshotGG: Jun 5 2005, 23:19


--------------------
College student/IT Assistant
Go to the top of the page
+Quote Post
de Mon
post Jun 5 2005, 23:54
Post #6





Group: Members
Posts: 474
Joined: 1-December 02
Member No.: 3940



QUOTE (HotshotGG @ Jun 5 2005, 02:16 PM)
... I say we all petition and send them a nasty e-mail or something pathetic  wink.gif. Who did they hire a Johnny Nothing off the street to do this? It says the test is "non-scientific" how would you expect anyone to take you seriously?
*


Got my vote!


--------------------
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech
Go to the top of the page
+Quote Post
Digisurfer
post Jun 6 2005, 00:42
Post #7





Group: Members
Posts: 371
Joined: 10-August 04
From: Canada
Member No.: 16174



Wonderful. You work your butt off sometimes (at least it feels that way) just to try an help folks you encounter around the net in understanding what lossy compression is really all about and why >proper< testing is important when deciding quality. There is already tons of misinformation floating around out there as it is, and I would say greater than 60% of the netizens out there have it completely wrong thanks to that fact. Then a totally irresponsible article like this one comes out and sets the world back even more, negating any progress we've clawed our way through. Personally I think this has nothing to do with which is better, or sounds more pleasant either, but a shady attempt and pusing more people to use Microsoft "product". It's feels like being pushed back into the dark ages. Oh well (heavy sigh) at least the good people of HA.org won't be fooled... right? wink.gif

Edit: Fixed bad typo some might find offensive lol.

This post has been edited by Digisurfer: Jun 6 2005, 00:44
Go to the top of the page
+Quote Post
westgroveg
post Jun 6 2005, 00:52
Post #8





Group: Members
Posts: 1236
Joined: 5-October 01
Member No.: 220



Weird that in "Cherub Rock," Smashing Pumpkins, WMA@128kbps scored higher than WMA@192kbps.

And I would have thought OGG@Q6 would be transparent in most cases, much better than WMA@128kbps too.

This post has been edited by westgroveg: Jun 6 2005, 00:53
Go to the top of the page
+Quote Post
Faelix
post Jun 6 2005, 02:10
Post #9





Group: Members
Posts: 54
Joined: 6-July 02
Member No.: 2496



This thread has already mentioned that wierd listening test.
Go to the top of the page
+Quote Post
guruboolez
post Jun 6 2005, 11:01
Post #10





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Busemann @ Jun 5 2005, 10:22 PM)
That the higher bitrate clips do worse seems to indicate that this is a test to find the  codec "which sounds best to you". Not most true to the original.

Bring in another crowd an the results could be the complete opposite.
*

Apparently, no:

QUOTE
He also periodically played the WAV file for reference anytime a juror requested.

http://www.download.com/Audio-Codec-Shoot-...20-5146328.html
Go to the top of the page
+Quote Post
tgoose
post Jun 6 2005, 11:54
Post #11





Group: Members
Posts: 407
Joined: 12-April 05
Member No.: 21399



If they couldn't differentiate between 128 and 192, then obviously 128 wins.
Go to the top of the page
+Quote Post
Pio2001
post Jun 6 2005, 11:58
Post #12


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



There was no mathematical analysis.
They probably did not hear any artifact in any file, and just thew random marks... Which explains why all codecs get nearly the same mark at all bitrates after summing up the results.

128 kbps got a better mark than 192 kbps 8 times, and 192 a better one than 128 12 times.
Go to the top of the page
+Quote Post
spoon
post Jun 6 2005, 12:20
Post #13


dBpowerAMP developer


Group: Developer (Donating)
Posts: 2758
Joined: 24-March 02
Member No.: 1615



It probabbly goes to show for anything > 128 Kbps 'normal' people are rubbish at listening tests.


--------------------
Spoon http://www.dbpoweramp.com
Go to the top of the page
+Quote Post
Pio2001
post Jun 7 2005, 11:41
Post #14


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



QUOTE (spoon @ Jun 6 2005, 01:20 PM)
It probabbly goes to show for anything > 128 Kbps 'normal' people are rubbish at listening tests.
*


They were no "normal people", they were selected as "folks with audio expertise and high standards ", from Download.com and MP3.com crews.
Go to the top of the page
+Quote Post
krabapple
post Jun 10 2005, 12:03
Post #15





Group: Members
Posts: 2515
Joined: 18-December 03
Member No.: 10538



For a test like this I'd really want some further descriptive statistics than just the average scores.

Also, I wonder how they enforced independence of listeners (who musnn't be allowed to influence each other) .

At least they say right up front that it wasn't scientific.

This post has been edited by krabapple: Jun 10 2005, 12:07
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th December 2014 - 23:40