IPB

Welcome Guest ( Log In | Register )

4 Pages V  « < 2 3 4  
Reply to this topicStart new topic
Pre-Test discussion, Looking for opinions.
rjamorim
post Jul 19 2003, 15:49
Post #76


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (den @ Jul 19 2003, 02:05 AM)
I agree in principle, but why QT in one and Nero in the other?  blink.gif  I must be missing something.

Because I'm facing lots of pressure from people wanting a listening test with Nero AAC since it didn't do so well. See it as a revenge attempt wink.gif (And no, I'm not talking about Ivan)

Besides, if I don't fuck up, the results of both tests would be comparable among themselves. You wouldn't be able to draw definitive conclusions (that would require same set of samples, same listener group, same listening conditions for each listener...), but you could at least have an idea.

QUOTE
Do we really need an anchor?


IMO we do, as I said before, to put things into perspective.

Low bitrate tests are even worse, they need two anchors. :B


@ff123: Have you found out if Blade is suitable for anchor?

Regards;

Roberto.


Edit: Yes, the mass-test would be the first.

This post has been edited by rjamorim: Jul 19 2003, 16:03


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ff123
post Jul 19 2003, 17:11
Post #77


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (rjamorim @ Jul 19 2003, 06:49 AM)
@ff123: Have you found out if Blade is suitable for anchor?

I ran through some comparisons, and here's how I would find things:

CODE
                  10kHz                   blade
41_30sec           easy                    very easy
atrain             moderate                moderate
beautyslept        easy                    easy
blackwater         moderate to difficult   easy
death2             moderate to difficult   very easy
flooressence       moderate to difficult   very easy
layla              moderate                easy
lifeshatters       moderate                very easy


I think blade would make a better anchor than 10 kHz, even though some samples may be harder to find problems in than others.

Note: Probably some people would find 10 kHz easy to detect on all samples. Some people might find it very hard to tell the difference on some. Maybe somebody else could ring in and try out a few?

ff123
Go to the top of the page
+Quote Post
rpop
post Jul 19 2003, 17:32
Post #78





Group: Super Moderator
Posts: 332
Joined: 20-May 03
From: Pittsburgh, USA
Member No.: 6718



So the masses test is the one scheduled to start Wednesday? How much later will the HA test start?


--------------------
[url=http://noveo.net/ph34r.htm]Happiness[/url] - The agreeable sensation of contemplating the misery of others.
Go to the top of the page
+Quote Post
rjamorim
post Jul 19 2003, 19:55
Post #79


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (rpop @ Jul 19 2003, 01:32 PM)
So the masses test is the one scheduled to start Wednesday?

Yes. Unless someone comes with a very good reason not to split the test in two.

(BTW, big thanks to Dibrom for pointing out the issue of the target audience)

QUOTE
How much later will the HA test start?


Much later. Before that, I plan to test codecs at 64kbps, and then do some vocodec tests with the help from JM Valin. Expect the HA test to happen somewhere around late this year, early next year (maybe).

With the extension test, it's enough 128kbps tests for now. :B

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ff123
post Jul 19 2003, 22:11
Post #80


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



Roberto,

Before you start the test, could you upload a test archive, so that I can doublecheck volume level and time alignment?

ff123
Go to the top of the page
+Quote Post
guruboolez
post Jul 19 2003, 23:34
Post #81





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (rjamorim @ Jul 19 2003, 01:54 AM)
The Masses test would compare QuickTime, Lame, Sonyīs Atrac3, WMA STD and Vorbis

The HA test would compare Nero AAC, Lame, Musepack, WMA PRO and Vorbis.

Just an apprehension : do you really consider QuickTime (and his codec) as a solution for the masses ? I know there's iTune (really same encoder : not sure), but it's for the next 6 months limited to MAC world (not really a mass-system). On the other side, QuickTime doesn't accept CDRipping or even batch encoding. And I prefer forget mass-tagging issues here... Mass or experts, QT isn't a useful tool. Last but not least, QT isn't intuitive for most people (save as... dosn't work : you need to export, and of course to know it).

Ahead encoder is so simple to use in comparison. And so many people have Nero installed on their computer. And popularity will certainly increase with the HE-AAC encoder, and its impressive quality at ~64 kbps.


If you really want to put both encoders on two different tests, I find more coherent to keep Nero VBR for the mass-test, and QT CBR for the HA one.

This post has been edited by guruboolez: Jul 19 2003, 23:38
Go to the top of the page
+Quote Post
bond
post Jul 19 2003, 23:39
Post #82





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



1) plz dont forget that the tests are about sound quality!!!!
and not about usability of the encoding tool or something else (which has nothing to do with the purpose mentioned above)...

2) codecs which were judged worse than another in a fair, scientific test (like quicktime - nero) are out at the tested bitrate (no need for a revenge until there are serious changes on the encoder side)

3) first of all everybody should ask himself:
"hey, what's the sense of a codec comparison @128kbps?"

-> people want to know which codec gives the best quality output at that bitrate

-> only the best codecs should be compared
-> quicktime, lame, vorbis, wma9 pro
(if you want to add additionally atrac3 or musepack or any other codec isnt really important to reach the goal of the comparison)

sorry to hear that rjamorim plans this (imho most) important comparison very late this year (or even next year) sad.gif

This post has been edited by bond: Jul 19 2003, 23:55


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
guruboolez
post Jul 19 2003, 23:51
Post #83





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (bond @ Jul 19 2003, 11:39 PM)
plz dont forget that the tests are about sound quality!!!!

and not about usability or something else (which has nothing to do with the purpose mentioned above)...

Indeed. But in my opinion, the difference between mass and 'expert' presuppose some ergonomy consideration.
Maybe the idea wasn't well explained :
- a PC only test (= HA test)
- a portable-use test (= mass test)

Even here, I have some reticences. I can't imagine someone filling an iPod with QT installed on his PC. Just try to encode one album only : it's a real pain.
A listening test should be useful, especially when practical purpose is in mind (call it portable-test or mass-test, it doesn't change anything).

When Doom9 performed nice codec comparison, he doesn't maximise perceptual quality with impossible avisyth script, and he's not using H264 encoder family - better theorical quality, but so slooooow... He have quality in mind, but doesn't forget to be in harmony with his readers behaviour and possibility

This post has been edited by guruboolez: Jul 19 2003, 23:52
Go to the top of the page
+Quote Post
bond
post Jul 20 2003, 00:03
Post #84





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



ups crosspost sad.gif

added point 2 and 3 in my last post...


ok, if you call it "portable-test" than the purpose of the test changed from "quality only" to "usability and quality", in this case i would add, like proposed from guruboolez, nero to the "mass-portable test" instead of quicktime...


am i the only one who thinks that the "ha-(only the best)-test" is more important and should be done first?

This post has been edited by bond: Jul 20 2003, 00:06


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
rjamorim
post Jul 20 2003, 00:17
Post #85


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello.

@Guruboolez: I understand your concern about QT not being really usable. But it has the best quality, it won the AAC test, and I'm not willing to fathom the criticism I expect will come from all sides about "WTF, QuickTime won the AAC test but you used Nero on the extension test though? Are you working for Ahead now, you bastard?".

You see, doom9 is a good example, but what is tying my hands now is the AAC test result.

@Bond: IMO, the HA test can't be conduced now. Even though it might be more important, still it would be wasting all the attention the AAC@128kbps test gathered.I plan to keet the HA test more "low profile", and not announce it everywhere else, at most at CD-rw.org

And no, this is a quality test. Even though usability is cool, it's an utterly subjective concept, and I want to keep these tests objective.

And, of course, I can be convinced to hurry up the HA test a little. wink.gif


@ff123: I'll contact you as soon as I have something ready (probably this tuesday, when I return home)
Have you found out the Blade offset when decoded by Lame?

Thank-you all.

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
bond
post Jul 20 2003, 00:37
Post #86





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



QUOTE (rjamorim @ Jul 20 2003, 01:17 AM)
"WTF, QuickTime won the AAC test but you used Nero on the extension test though? Are you working for Ahead now, you bastard?"

laugh.gif

QUOTE
IMO, the HA test can't be conduced now. Even though it might be more important, still it would be wasting all the attention the AAC@128kbps test gathered.I plan to keet the HA test more "low profile", and not announce it everywhere else, at most at CD-rw.org

i dont think that it will waste attention
-> the only "big" difference between the two tests is wma9 pro and std
-> the most important purpose for using an audio codec is storing music clips on the hd (portables are much less important imho compared to storing on hd (especially @128) and i dont want to know how many people from the masses reencode their stuff to lower bitrates anyways before using it in a portable)...

-> so if wma9 std will be replaced by the pro version, this will not lead to a loss in attention imo even the dumbest user will think: "hey, of course the pro version is better than the standard version and rjamorim also used the pro version, so there can be no doubt that its better, lets drop wma9 std @128"

-> i dont think that it will be a fault to let the pro version out on the masses (and the wma9 std codec will be important in the 64kbps test anyways (especially for portables))

QUOTE
And, of course, I can be convinced to hurry up the HA test a little.

would be really great

This post has been edited by bond: Jul 20 2003, 00:49


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
guruboolez
post Jul 20 2003, 01:28
Post #87





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (rjamorim @ Jul 20 2003, 12:17 AM)
@Guruboolez: I understand your concern about QT not being really usable. But it has the best quality, it won the AAC test, and I'm not willing to fathom the criticism I expect will come from all sides about "WTF, QuickTime won the AAC test but you used Nero on the extension test though? Are you working for Ahead now, you bastard?".

What I tried to say is : if you really want to include Nero encoding in one of both test, put it in a "mass" or "popular" part.
I really want to see QT@128 compete with other formats. It was designed as the winner, confirming the positive impressions I had before the public test, and I felt again during this test. There are no reason for me to eliminate QuickTime. On contrary : I have more difficulties to conceive good reasons for giving to Nero encoder a second chance - especially on a dedicate "HA test", - designed to be the high-quality-and-quality-only test. I like Ivan's work ; I know that Nero's codec was handicaped by a forced CBR encoding during test ; I'm aware that some progress were made since first test... but without public testing, proving its superiority on QT 6.3 on most samples, introducing Ahead in the next test is for me something really odd.

This post has been edited by guruboolez: Jul 20 2003, 01:56
Go to the top of the page
+Quote Post
ff123
post Jul 22 2003, 04:20
Post #88


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (rjamorim @ Jul 19 2003, 03:17 PM)
@ff123: I'll contact you as soon as I have something ready (probably this tuesday, when I return home)
Have you found out the Blade offset when decoded by Lame?

Yes, the offset is 0

ff123
Go to the top of the page
+Quote Post
rjamorim
post Jul 23 2003, 23:16
Post #89


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (ff123 @ Jul 22 2003, 12:20 AM)
Yes, the offset is 0

ff123

Thanks smile.gif



OK, so here's the deal:

-Sonicstage almost fried my Windows installation (and I used to think QuickTime is buggy). Also, I couldn't find out if I should rather use the SonicStage encoder or the Real Producer. As I said before, they are targeted at different purposes, and their encodings differ. I couldn't chose the portable one or the multipurpose one, and noone seemed to help in that decision.

Bottom line: Atrac3 is out.

-Given that Atrac3 is out, it's being replaced with Musepack and I won't split the tests.

-WMA standard won't be tested because, knowing Microsoft, it'll be quickly ditched and replaced with it's superior brother. So, only WMA pro wil be tested. Lack of hardware support isn't an argument, if that was an argument, MPC and Vorbis would be out too. I can expect that, in some months, there'll be more hardware supporting WMA Pro than Vorbis, or AAC.

-What about Nero? I didn't think about it yet, so don't ask. For the time being, it won't be tested (at least by me) wink.gif

-most important reason not to split tests (and I quote Guruboolez)
"Test is boring. Two tests will be a real pain."

Test starts in about one hour, if all goes right while uploading.

Best regards;

Moi.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
bond
post Jul 23 2003, 23:27
Post #90





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



QUOTE (rjamorim @ Jul 24 2003, 12:16 AM)
Atrac3 is out ... WMA standard won't be tested ... not to split tests

good news,
(i guess we will see wma9 std and atrac/realaudio again in the 64kbps test to come anyway...)

QUOTE
What about Nero? For the time being, it won't be tested

so quicktime is used instead? hm, i think so biggrin.gif

QUOTE
Test starts in about one hour, if all goes right while uploading.

gimme, gimme laugh.gif


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
rjamorim
post Jul 24 2003, 00:32
Post #91


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (bond @ Jul 23 2003, 07:27 PM)
gimme, gimme  laugh.gif

Of course. biggrin.gif

http://rarewares.hydrogenaudio.org/test/12...esentation.html


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
den
post Jul 24 2003, 01:34
Post #92





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



Good stuff Roberto. I think you have made the right decisions. B)

QUOTE
Sonicstage almost fried my Windows installation (and I used to think QuickTime is buggy).


I also had a near death experience with Sonicstage blink.gif , hence my continuing use of OpenMG Jukebox, which has the same internals as Sonicstage, but uses a slightly less funky/sexy interface, and a more reliable install routine.

Welcome to the world of NetMD, Sony style. :x

Looking forward to this test, and will get my listening done over the weekend...

Den.


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
rjamorim
post Jul 24 2003, 02:58
Post #93


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (den @ Jul 23 2003, 09:34 PM)
Good stuff Roberto. I think you have made the right decisions.  B)

Thanks smile.gif

QUOTE
I also had a near death experience with Sonicstage  blink.gif , hence my continuing use of OpenMG Jukebox, which has the same internals as Sonicstage, but uses a slightly less funky/sexy interface, and a more reliable install routine.

Welcome to the world of NetMD, Sony style. :x


True. tongue.gif

maybe I'll test it one of these days. In this occasion, I would probably create a Win98 partition with VirtualPC and install SonicStage inside, and later delete it. But it sure as hell has a very low priority now.

QUOTE
Looking forward to this test, and will get my listening done over the weekend...


Great, looking forward to your test results.

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post

4 Pages V  « < 2 3 4
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th July 2014 - 04:42