IPB

Welcome Guest ( Log In | Register )

7 Pages V  « < 3 4 5 6 7 >  
Reply to this topicStart new topic
64 kbps listening test 2005, Pre-test thread
Sebastian Mares
post Mar 30 2005, 20:41
Post #101





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (guruboolez @ Mar 30 2005, 09:24 PM)
As I said, I was not fully satisfied by this test (too fast, too imprecise). If the collective test doesn't start in the next days, I think I could test CBR and VBR again, without WMApro this time, and with ABX phase in order to be sure that difference were audible.
*


Well, I guess it will take at least one more week for the test to start. Apple's HE-AAC encoder isn't even out yet. smile.gif

Also, again, Vorbis tests are welcome to help deciding which encoder to use. smile.gif


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Busemann
post Mar 30 2005, 21:05
Post #102





Group: Members
Posts: 730
Joined: 5-January 04
Member No.: 10970



QUOTE (Sebastian Mares @ Mar 30 2005, 11:41 AM)
Well, I guess it will take at least one more week for the test to start. Apple's HE-AAC encoder isn't even out yet. smile.gif
*



I reckon HE-AAC is still a bit off, definitely more than 1 week.
Go to the top of the page
+Quote Post
slippyC
post Mar 30 2005, 21:44
Post #103





Group: Members
Posts: 65
Joined: 1-January 04
Member No.: 10845



I know this is a little late in the thread and I may have missed it, but I think you ought to do both WMA and WMAPro.

Reason being is because WMA is widespread, but WMAPro is superior to Standard. It would keep people from becoming confused and still giving the WMAPro codec a fair chance(to keep the WMA fanatics off your back).
Go to the top of the page
+Quote Post
rjamorim
post Mar 30 2005, 22:09
Post #104


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Busemann @ Mar 30 2005, 05:05 PM)
I reckon HE-AAC is still a bit off, definitely more than 1 week.
*


Rumour has it that Tiger will be released mid-april. Of course, Apple rumours are NEVER trustworthy.

QUOTE (slippyC @ Mar 30 2005, 05:44 PM)
Reason being is because WMA is widespread, but WMAPro is superior to Standard.  It would keep people from becoming confused and still giving the WMAPro codec a fair chance(to keep the WMA fanatics off your back).
*


I think the level of usefulness of WMA Pro at 64kbps is too small. Microsoft clearly sends you the message that they only want you using it above 128kbps (in CBR mode at least). Forcing it to go lower with VBR results in too random bitrates that would make the test biased for one side or the other.

Also, it's worth remembering such low bitrates are useful for a) limited memory hardware or b) streaming. a) is pointless because no portable player supports WMA Pro. b) makes no sense because VBR doesn't mix well with streaming.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
slippyC
post Mar 31 2005, 04:50
Post #105





Group: Members
Posts: 65
Joined: 1-January 04
Member No.: 10845



QUOTE (rjamorim @ Mar 30 2005, 01:09 PM)
I think the level of usefulness of WMA Pro at 64kbps is too small. Microsoft clearly sends you the message that they only want you using it above 128kbps (in CBR mode at least). Forcing it to go lower with VBR results in too random bitrates that would make the test biased for one side or the other.

Also, it's worth remembering such low bitrates are useful for a) limited memory hardware or b) streaming. a) is pointless because no portable player supports WMA Pro. b) makes no sense because VBR doesn't mix well with streaming.
*



Yea, I don't know of any portable support. If I remember right someone on this board said that MS never intended it to be.

Anyway, was just a suggestion for some folks concerns. I don't use WMAPro, have messed with it a little in the past, so didn't matter much to me.
Go to the top of the page
+Quote Post
guruboolez
post Mar 31 2005, 08:46
Post #106





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Suggestion: if you plan to publish a bitrate table, and in order to avoid (at least to limit) the usual discussion about 'fairness', 'apples & orange', etc... why not publishing a bitrate table of the full encoded song instead of the short samples (~20 sec), which are probably higher.

It's not really easy (you have to download full samples, or have to ask to people submitting samples to encode their files for you and then report the exact value), but it could be more representative of the bitrate of very short samples. On visual tests (video, cf. doom9.org), we have the bitrate for the full movie, and not for the selected frames. What do you think?
Go to the top of the page
+Quote Post
Gabriel
post Mar 31 2005, 08:55
Post #107


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Regarding the low anchor, I would personnally like Lame @abr.
My own justification is:
*that would probably not change that much the notation of the low anchor, and anyway the purpose of the low anchor is to be low.
*that would allow me to have fine input regarding Lame's true performance @64k compared to modern codecs
Go to the top of the page
+Quote Post
AtaqueEG
post Mar 31 2005, 08:59
Post #108





Group: Members (Donating)
Posts: 1336
Joined: 18-November 01
From: Celaya, Guanajuato
Member No.: 478



QUOTE (guruboolez @ Mar 31 2005, 01:46 AM)
Suggestion: if you plan to publish a bitrate table, and in order to avoid (at least to limit) the usual discussion about 'fairness', 'apples & orange', etc... why not publishing a bitrate table of the full encoded song instead of the short samples (~20 sec), which are probably higher.

It's not really easy (you have to download full samples, or have to ask to people submitting samples to encode their files for you and then report the exact value), but it could be more representative of the bitrate of very short samples. On visual tests (video, cf. doom9.org), we have the bitrate for the full movie, and not for the selected frames. What do you think?
*


This is a great idea.
And it would give you a "real life" scenario, showing the actual behavior of the codec with full songs, and how it would act on such tracks.


--------------------
I'm the one in the picture, sitting on a giant cabbage in Mexico, circa 1978.
Reseñas de Rock en Español: www.estadogeneral.com
Go to the top of the page
+Quote Post
nyarlathotep
post Mar 31 2005, 16:24
Post #109





Group: Members
Posts: 412
Joined: 13-April 03
From: Haut-Médoc FRA
Member No.: 5943



QUOTE (guruboolez @ Mar 24 2005, 09:50 AM)
• I think that the current scale isn't really really suited to a 64 kbps and the expected distortions.
Artifacts “perceptible but not annoying” (4.0) are maybe not very common at this bitrate. And few encoders are able to reproduce (in my opinion) a sound with only "slightly annoying" (3.0) difference at 64 kbps. It's possible to change the corresponding scale with schnofler's abc/hr, and I wonder if it's not worth to think about it. If I remember correctly, the average notation I gave to most encoders during the 32 kbps was inferior to 1.5/5  unsure.gif

I second that suggestion. During the 32 kbps test, I gave most sample only between 1 and 2 because then I decided to follow the comments near the scale while (it seems that) a lot of people use the whole scale without taking into account the comments '“perceptible but not annoying” and so on...).

I mean that the situation should be clarified: should we use the whole scale or follow the comments?

This post has been edited by nyarlathotep: Mar 31 2005, 16:27
Go to the top of the page
+Quote Post
Sebastian Mares
post Mar 31 2005, 16:54
Post #110





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (nyarlathotep @ Mar 31 2005, 05:24 PM)
I mean that the situation should be clarified: should we use the whole scale or follow the comments?
*


I will modify to the labels to excellent - poor.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Supernaut
post Mar 31 2005, 19:30
Post #111





Group: Members
Posts: 136
Joined: 9-April 04
Member No.: 13359



QUOTE (Sebastian Mares @ Mar 31 2005, 04:54 PM)
I will modify to the labels to excellent - poor.

Perhaps you could post your choice of the labels so we can discuss and perhaps improve them, in order to make them more descriptive and as unambiguous as possible? I'm not sure "good" and "excellent" labels would help me make my results on scale with others' results...
Go to the top of the page
+Quote Post
Busemann
post Apr 1 2005, 00:19
Post #112





Group: Members
Posts: 730
Joined: 5-January 04
Member No.: 10970



QUOTE (rjamorim @ Mar 30 2005, 01:09 PM)
Rumour has it that Tiger will be released mid-april. Of course, Apple rumours are NEVER trustworthy.



Tiger has gone GM, the question is whether HE-AAC will be included in QT 7 after all..

This post has been edited by Busemann: Apr 1 2005, 00:21
Go to the top of the page
+Quote Post
kurtnoise
post Apr 1 2005, 17:52
Post #113





Group: Members
Posts: 326
Joined: 26-June 02
From: Aix-en-Provence
Member No.: 2400



Sebastian...I don't know if you are already begun the preliminaries for these tests, but may I suggest to add a new candidate : Enhanced AAC+ v2 ??

Brandon Siegel has updated its CLI encoder for dbPowerAmp. It can now reach up to 64 kbps for stereo files. Check this thread directly on dBAmp forum for more infos.


Another question : have you got enough samples or can we post new ones ?
Go to the top of the page
+Quote Post
guruboolez
post Apr 1 2005, 17:59
Post #114





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



HE-AAC v.2 = AAC + SBR (v.1) + Parametric Stereo (=> v.2). According to some tests requested by Ivan ~one year ago, Parametric Stereo have a negative impact at "high" bitrate (> 40 kbps IIRC). Of course, it was noticed on Nero AAC encoder, and it doesn't necessary mean than other encoders have similar reactions. Therfore, it could be worth to test.


To be entirely honest, I don't really like the idea of testing two HE-AAC encoders (Nero & Apple) in a same multiformat test. I don't like it, but I understand the reason I think. I would rather make preliminary tests, in order to choose the best challenger for each format (as we do for wma, vorbis...). There are many possible contenders:

- Nero AAC VBR & CBR (does VBR produce stable quality at low bitrate?)
- Apple VBR & CBR (same comment)
- Coding Technologie (in Real products) AAC
- the one pointed by kurtnoise.

But to make this possible, we need people to test them first.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 1 2005, 18:19
Post #115





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (kurtnoise @ Apr 1 2005, 06:52 PM)
Sebastian...I don't know if you are already begun the preliminaries for these tests, but may I suggest to add a new candidate : Enhanced AAC+ v2 ??

Brandon Siegel has updated its CLI encoder for dbPowerAmp. It can now reach up to 64 kbps for stereo files. Check this thread directly on dBAmp forum for more infos.


Another question : have you got enough samples or can we post new ones ?
*


Well, with Apple and Nero, the two most popular HE-AAC encoders, I have enough HE-AAC encoders covered. smile.gif

Regarding samples, I have enough now, thank you. It's a bit hard to decide which ones to use, since almost all of them are more or less killer samples.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 1 2005, 18:22
Post #116





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (guruboolez @ Apr 1 2005, 06:59 PM)
To be entirely honest, I don't really like the idea of testing two HE-AAC encoders (Nero & Apple) in a same multiformat test. I don't like it, but I understand the reason I think. I would rather make preliminary tests, in order to choose the best challenger for each format (as we do for wma, vorbis...). There are many possible contenders:

- Nero AAC VBR & CBR (does VBR produce stable quality at low bitrate?)
- Apple VBR & CBR (same comment)
- Coding Technologie (in Real products) AAC
- the one pointed by kurtnoise.

But to make this possible, we need people to test them first.
*


I guess you are right, but who wants to conduce another pre-test? Look at how many people tested Vorbis.

Edit: By the way... Regarding WMA, I guess I will stick to Standard since Pro isn't really "tuned" for such bit-rates and as Roberto said, 64 kbps is used for either streaming or portable usage. None of those areas are covered well by Pro.

This post has been edited by Sebastian Mares: Apr 1 2005, 18:32


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
guruboolez
post Apr 1 2005, 18:28
Post #117





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Sebastian Mares @ Apr 1 2005, 06:22 PM)
I guess you are right, but who wants to conduce another pre-test? Look at how many people tested Vorbis.
*


I had this in mind when I said "I don't like it, but I understand the reason I think" smile.gif
Preliminary collective tests are possible, without someone to conduce it from A to Z. Remember last 128 kbps listening test: there were many choices for vorbis (CVS, aoTuV, Quantum Knot, Modest Tuning), and the final decision was based upon personal tests. We know the result, more than positive for Vorbis smile.gif
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 1 2005, 18:36
Post #118





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (guruboolez @ Apr 1 2005, 07:28 PM)
QUOTE (Sebastian Mares @ Apr 1 2005, 06:22 PM)
I guess you are right, but who wants to conduce another pre-test? Look at how many people tested Vorbis.
*


I had this in mind when I said "I don't like it, but I understand the reason I think" smile.gif
Preliminary collective tests are possible, without someone to conduce it from A to Z. Remember last 128 kbps listening test: there were many choices for vorbis (CVS, aoTuV, Quantum Knot, Modest Tuning), and the final decision was based upon personal tests. We know the result, more than positive for Vorbis smile.gif
*



Yes, but who has the time to spend on testing? I don't think I can justify a winner based on five or six test results. sad.gif

Edit: BTW, everyone is free to conduce pre-tests.

This post has been edited by Sebastian Mares: Apr 1 2005, 18:37


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
guruboolez
post Apr 1 2005, 18:59
Post #119





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Sebastian Mares @ Apr 1 2005, 06:36 PM)
I don't think I can justify a winner based on five or six test results. sad.gif
*


I can't find the exact thread, but if I remember correctly, there were no more persons to test various fork of Vorbis in order to select the good one last spring.

QUOTE
Yes, but who has the time to spend on testing?

Time is not the problem. Motivation is probably a greater one. I don't think that many people will be interested to test HE-AAC implementation that are not used otherwise.

Nevertheless, removing one redundant challenger could make the test easier (each additionnal codec makes evaluation and the establishment of the hierarchy harder). There are currently 7 contenders (including anchor): that's very much, especially for untrained people.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 1 2005, 19:09
Post #120





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (guruboolez @ Apr 1 2005, 07:59 PM)
Nevertheless, removing one redundant challenger could make the test easier (each additionnal codec makes evaluation and the establishment of the hierarchy harder). There are currently 7 contenders (including anchor): that's very much, especially for untrained people.
*


Well, the most I can do is dump ATRAC3+ and if enough tests are available, one of the HE-AAC encoders.
The reason why I wanted both HE-AAC encoders to be featured is that a lot of work was done to improve the Nero encoder and it is also interesting to see how Apple performs against the big competitor (so there is a direct comparison).


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 1 2005, 19:12
Post #121





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (guruboolez @ Apr 1 2005, 07:59 PM)
I can't find the exact thread, but if I remember correctly, there were no more persons to test various fork of Vorbis in order to select the good one last spring.
*


This?


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
guruboolez
post Apr 1 2005, 19:23
Post #122





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Yes, this one. I've found the exact post I had in mind. Roberto had only 5 set of results (+ 2 partial results) to make a decision. Also note that aoTuV beta 1 was pre-tested, and that the non-tested aoTuV beta2 was included in the final collective test... and won (tied with mpc). Taking some risks is not necessary a bad thing tongue.gif

This post has been edited by guruboolez: Apr 1 2005, 19:24
Go to the top of the page
+Quote Post
rudefyet
post Apr 13 2005, 02:48
Post #123





Group: Members
Posts: 229
Joined: 29-December 03
From: Columbus, Ohio
Member No.: 10785



QUOTE (guruboolez @ Apr 1 2005, 08:59 AM)
- Nero AAC VBR & CBR (does VBR produce stable quality at low bitrate?)


From personal tests the "fast" mode on Nero causes some serious problems/distortions around 96kbps or lower in LC-AAC and somewhere below 64kbps it causes trouble on HE-AAC (32kbps HE-AAC in fast mode sounded horrific! High mode sounded much better)

high mode with the "streaming" profile gives good quality 64kbps HE-AAC, fast mode sounds similar at 64kbps, but the bitrates only average around ~50-55kbps, i haven't done any ABxing to determine any quality differences though

This post has been edited by rudefyet: Apr 13 2005, 02:50
Go to the top of the page
+Quote Post
negritot
post Apr 13 2005, 06:42
Post #124





Group: Members
Posts: 209
Joined: 2-January 04
Member No.: 10890



So Quicktime 7 will be widely available in a little over two weeks. Is that enough time to narrow down the contenders? And can anyone verify that Quicktime 7 includes an HE-AAC encoder? AAC isn't even mentioned on the preview pages for Quicktime 7.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 13 2005, 07:32
Post #125





Group: Members
Posts: 3630
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (negritot @ Apr 13 2005, 07:42 AM)
So Quicktime 7 will be widely available in a little over two weeks. Is that enough time to narrow down the contenders? And can anyone verify that Quicktime 7 includes an HE-AAC encoder? AAC isn't even mentioned on the preview pages for Quicktime 7.
*


Well, I am still waiting for some Vorbis tests. Now that AoTuV PB 4 is out, somebody might want to test that against Xiph 1.1 and Archer.
Regarding HE-AAC tests, I doubt that anyone would like to test on their own. Also, I would rather get the test started pretty soon after Apple releases their HE-AAC encoder, so there won't be much room (time) for pre-tests. Therefore, I guess I will stick to my first decision featuring both codecs in the main test.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post

7 Pages V  « < 3 4 5 6 7 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 23rd September 2014 - 05:53