IPB

Welcome Guest ( Log In | Register )

7 Pages V   1 2 3 > »   
Reply to this topicStart new topic
64 kbps listening test 2005, Pre-test thread
Sebastian Mares
post Mar 22 2005, 15:54
Post #1





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Greetings!

I am planning to conduce a multi-format 64 kbps listening test under Roberto's supervision. The exact date for the beginning of the test is not set, since I am waiting for Apple to release their HE-AAC encoder sometime in April (according to the rumours).

So far, here is what I was thinking about. Feedback and suggestions are welcome.

Codecs:
  • Nero HE-AAC
  • Apple HE-AAC
  • Ogg Vorbis
  • Windows Media Audio
  • LAME 128 kbps as high anchor
  • Fraunhofer 64 kbps as low anchor
  • mp3PRO maybe
Nero and Apple HE-AAC will be included for sure.
What I don't really know is which Vorbis version to use - either AoTuV3 or 1.1. Same applies to WMA - either Standard due to compatibility or Professional for quality. Last but not least, should mp3PRO be tested again as reference (since there aren't any changes from version used in Roberto's last test)?

As you can see, there will be 6 or 7 (depending on mp3PRO) codecs.

Samples:
  • DaFunk
  • EnolaGay
  • experienca
  • gone
  • Illinois
  • mybloodrusts
  • NewYorkCity
  • riteofspring
  • Scars
I am planning to use 18 samples - 9 from Roberto's old listening test and 9 additional ones. What do you guys suggest?

The results will be calculated in the same way Roberto's results were calculated.

So, what are you thoughts? biggrin.gif

Regards,
Sebastian

Edit: Spelling and typo in the number of samples planned to use.

This post has been edited by Sebastian Mares: Mar 22 2005, 16:00


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Sebastian Mares
post Mar 22 2005, 15:58
Post #2





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



As difficult sample, I would suggest Kraftwerk to replace Waiting (which was used in the last test and is not missing).


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
sehested
post Mar 22 2005, 16:15
Post #3





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



Good initiative. I will look forward to the test.

I suggest to leave mp3PRO out, since nobody use it.

As for WMA I would recommend using WMA std. It is more widespread than PRO and is what most people refer to as WMA. People already have a hard time distinguishing between std. and PRO so choosing PRO is likely to lead to misinterpretation of the test results. As an alternative one could consider including both WMA std and WMA PRO, anyway my vote on WMA std.

What about using LAME 3.97a8 as low anchor?
Go to the top of the page
+Quote Post
Latexxx
post Mar 22 2005, 16:22
Post #4


A/V Moderator


Group: Members
Posts: 858
Joined: 12-May 03
From: Finland
Member No.: 6557



I would take Sony's Atrac3plus @ 64 kbps. It has shown some potential with non-problem samples and it would be great to compare it to other formats.
Go to the top of the page
+Quote Post
PaleGreen
post Mar 22 2005, 16:24
Post #5





Group: Members
Posts: 16
Joined: 12-January 05
Member No.: 19004



I'd like to see both WMA Standard & Pro included.
Go to the top of the page
+Quote Post
Latexxx
post Mar 22 2005, 16:26
Post #6


A/V Moderator


Group: Members
Posts: 858
Joined: 12-May 03
From: Finland
Member No.: 6557



QUOTE (PaleGreen @ Mar 22 2005, 05:24 PM)
I'd like to see both WMA Standard & Pro included.
*

I wouldn't include wma pro because no internet radio uses it and it would only confuse people.
Go to the top of the page
+Quote Post
rjamorim
post Mar 22 2005, 16:32
Post #7


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Latexxx @ Mar 22 2005, 12:26 PM)
QUOTE (PaleGreen @ Mar 22 2005, 05:24 PM)
I'd like to see both WMA Standard & Pro included.
*

I wouldn't include wma pro because no internet radio uses it and it would only confuse people.
*



I wouldn't include WMA standard because it has already been tested and, as far as I know, it hasn't changed much, if any, since tongue.gif


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
sehested
post Mar 22 2005, 16:34
Post #8





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (rjamorim @ Mar 22 2005, 07:32 AM)
I wouldn't include WMA standard because it has already been tested and, as far as I know, it hasn't changed much, if any, since tongue.gif
*


May I then suggest we use it as reference?
Go to the top of the page
+Quote Post
rjamorim
post Mar 22 2005, 16:35
Post #9


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (sehested @ Mar 22 2005, 12:34 PM)
May I then suggest we use it as reference?
*


FhG 64kbps (low anchor) didn't change much either, AFAIK.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
sehested
post Mar 22 2005, 16:37
Post #10





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (rjamorim @ Mar 22 2005, 07:35 AM)
QUOTE (sehested @ Mar 22 2005, 12:34 PM)
May I then suggest we use it as reference?
*


FhG 64kbps (low anchor) didn't change much either, AFAIK.
*


In that case we can leave out WMA completely
Go to the top of the page
+Quote Post
Gabriel
post Mar 22 2005, 16:39
Post #11


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Perhaps Lame could be used as low anchor?
Go to the top of the page
+Quote Post
Latexxx
post Mar 22 2005, 16:48
Post #12


A/V Moderator


Group: Members
Posts: 858
Joined: 12-May 03
From: Finland
Member No.: 6557



QUOTE (sehested @ Mar 22 2005, 05:37 PM)
QUOTE (rjamorim @ Mar 22 2005, 07:35 AM)
QUOTE (sehested @ Mar 22 2005, 12:34 PM)
May I then suggest we use it as reference?
*


FhG 64kbps (low anchor) didn't change much either, AFAIK.
*


In that case we can leave out WMA completely
*


If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?
Go to the top of the page
+Quote Post
guruboolez
post Mar 22 2005, 16:49
Post #13





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Latexxx @ Mar 22 2005, 04:22 PM)
I would take Sony's Atrac3plus @ 64 kbps. It has shown some potential with non-problem samples and it would be great to compare it to other formats.
*

I second that suggestion. Atrac3plus wasn't tested yet (only atrac3@132 kbps). It could be very interesting to see how will perform this format, which correspond (according to optimistic estimations made by Sony) to atrac@132 and to atrac@292 kbps:

http://www.mdfr.com/faqs/atrac_roadmap_small.jpg

But it's necessary to upload them as lossless files (flac or something else).
Go to the top of the page
+Quote Post
Thoifl
post Mar 22 2005, 16:53
Post #14





Group: Members
Posts: 4
Joined: 27-August 04
Member No.: 16570



QUOTE
What I don't really know is which Vorbis version to use - either AoTuV3

I would take AoTuV3, since it should be tweaked for low bitrates and is little testet.
Go to the top of the page
+Quote Post
rjamorim
post Mar 22 2005, 16:55
Post #15


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Latexxx @ Mar 22 2005, 12:48 PM)
If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?
*


Good point. But then again, most slashdotters can go .... themselves, for all that I really care...

QUOTE (guruboolez @ Mar 22 2005, 12:49 PM)
QUOTE (Latexxx @ Mar 22 2005, 04:22 PM)
I would take Sony's Atrac3plus @ 64 kbps. It has shown some potential with non-problem samples and it would be great to compare it to other formats.
*

I second that suggestion. Atrac3plus wasn't tested yet (only atrac3@132 kbps). It could be very interesting to see how will perform this format, which correspond (according to optimistic estimations made by Sony) to atrac@132 and to atrac@292 kbps:

http://www.mdfr.com/faqs/atrac_roadmap_small.jpg

But it's necessary to upload them as lossless files (flac or something else).
*



Well, same would have to happen with MP3pro, that is another alternative for 7th codec.

So, I guess the discussion is about of wether Atrac3+ or MP3pro should be tested.

I already feel sorry for Sebastian if people choose Atrac3+ rolleyes.gif

This post has been edited by rjamorim: Mar 22 2005, 16:58


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
DreamTactix291
post Mar 22 2005, 16:56
Post #16





Group: Members (Donating)
Posts: 552
Joined: 9-June 04
From: A place long since forgotten...
Member No.: 14572



QUOTE (Sebastian Mares @ Mar 22 2005, 08:54 AM)
What I don't really know is which Vorbis version to use - either AoTuV3 or 1.1. Same applies to WMA - either Standard due to compatibility or Professional for quality.
Since aoTuV b2 (which 1.1 is essentially) has been fairly well tested it might be interesting to see how the relative newcomer aoTuV b3 fares IMO. Even worse possible scenario of regression I can't see it being really that much worse than 1.1 (which I doubt it is worse but only testing can prove either way).

As far as WMA I'd go standard. Sure it's not as high quality as pro but I've never even come across a WMA Pro file I didn't make myself in the wild.


--------------------
Nero AAC 1.5.1.0: -q0.45
Go to the top of the page
+Quote Post
rjamorim
post Mar 22 2005, 16:56
Post #17


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Gabriel @ Mar 22 2005, 12:39 PM)
Perhaps Lame could be used as low anchor?
*


Dunno... wouldn't FhG be more recommended, since it is, at least in theory, better at low bitrates thanks to IS?


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
sTisTi
post Mar 22 2005, 16:58
Post #18





Group: Members
Posts: 385
Joined: 25-June 04
Member No.: 14895



QUOTE (Latexxx @ Mar 22 2005, 07:48 AM)
If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?
*

WMA std should definitely be included. I'm sure it will fail badly against Vorbis and HE AAC, so this test will be one more evidence against Microsoft's ridiculous claims and demonstrate the low quality of WMA compared to other modern codecs.


--------------------
Proverb for Paranoids: "If they can get you asking the wrong questions, they don't have to worry about answers."
-T. Pynchon (Gravity's Rainbow)
Go to the top of the page
+Quote Post
guruboolez
post Mar 22 2005, 16:59
Post #19





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



atrac3plus could be tested over mp3pro, for some reasons:
- mp3pro was tested previously, and hasn't progress since.
- atrac3plus wasn't tested
- there are more device playing atrac3plus than mp3pro (all modern HiMD units, many CD players, Jukebox and Sony flash USB devices).
- atrac3plus have more supporters (I'd use another word to describe them...) than mp3pro, which has progressively lost importance with time and competition.
Go to the top of the page
+Quote Post
Latexxx
post Mar 22 2005, 17:00
Post #20


A/V Moderator


Group: Members
Posts: 858
Joined: 12-May 03
From: Finland
Member No.: 6557



QUOTE (rjamorim @ Mar 22 2005, 05:55 PM)
I already feel sorry for Sebastian if people choose Atrac3+  rolleyes.gif
*

If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
Go to the top of the page
+Quote Post
guruboolez
post Mar 22 2005, 17:01
Post #21





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (rjamorim @ Mar 22 2005, 04:56 PM)
QUOTE (Gabriel @ Mar 22 2005, 12:39 PM)
Perhaps Lame could be used as low anchor?
*


Dunno... wouldn't FhG be more recommended, since it is, at least in theory, better at low bitrates thanks to IS?
*


MP3 at 64 kbps must be a low anchor. It doesn't matter to use lame or fhg. All we need is a encoder which sound worse than all other competitors.
Testing lame at 128 and at 64 kbps is IMO a good idea.
Go to the top of the page
+Quote Post
guruboolez
post Mar 22 2005, 17:03
Post #22





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Latexxx @ Mar 22 2005, 05:00 PM)
QUOTE (rjamorim @ Mar 22 2005, 05:55 PM)
I already feel sorry for Sebastian if people choose Atrac3+  rolleyes.gif
*

If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
*


I must warn you that basic MD lovers won't consider results as significant unless you've made encoding directly from device, and not from SonicStage. Hardware encoding is often considered as better than software encodings (placebo or not: I don't know blind listening tests were never done to my knowledge).
Go to the top of the page
+Quote Post
rjamorim
post Mar 22 2005, 17:03
Post #23


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Latexxx @ Mar 22 2005, 01:00 PM)
If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
*


Ah, that's good news, indeed. But he'll still have to install SonicStage. That flaming piece of poo ruined my win2000 installation.

QUOTE
I must warn you that basic MD lovers won't consider results as significant unless you've made encoding directly from device, and not from SonicStage. Hardware encoding is often considered as better than software encodings (placebo or not: I don't know — blind listening tests were never done to my knowledge).


That is certifiable bollocks. Hardware encoding has no reason to be better than software encoding, quite the opposite: hardware encoding faces limitations related to battery consumption, real-time processing on underpowered DSPs, integer-only processing, etc, etc.

This post has been edited by rjamorim: Mar 22 2005, 17:06


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Gabriel
post Mar 22 2005, 17:06
Post #24


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
Dunno... wouldn't FhG be more recommended, since it is, at least in theory, better at low bitrates thanks to IS?

I am not sure if the difference would be that big between FhG and 3.97b.

Anyway, whatever the choosen anchors, I'd like the graphs of results mentionning "low anchor" and "high anchor" instead of the codec name.
Go to the top of the page
+Quote Post
Latexxx
post Mar 22 2005, 17:09
Post #25


A/V Moderator


Group: Members
Posts: 858
Joined: 12-May 03
From: Finland
Member No.: 6557



QUOTE (rjamorim @ Mar 22 2005, 06:03 PM)
QUOTE (Latexxx @ Mar 22 2005, 01:00 PM)
If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
*


Ah, that's good news, indeed. But he'll still have to install SonicStage. That flaming piece of poo ruined my win2000 installation.

That is certifiable bollocks. Hardware encoding has no reason to be better than software encoding, quite the opposite: hardware encoding faces limitations related to battery consumption, real-time processing on underpowered DSPs, integer-only processing, etc, etc.
*



I have a working SS installation if that matters.

There are some hardware atrac3 vs SS atrac3 abx threads somewhere.
Go to the top of the page
+Quote Post

7 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th September 2014 - 10:06