IPB

Welcome Guest ( Log In | Register )

5 Pages V  < 1 2 3 4 5 >  
Reply to this topicStart new topic
64kbps public listening test, FINISHED
rjamorim
post Sep 22 2003, 15:43
Post #51


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (guruboolez @ Sep 22 2003, 11:21 AM)
Is it possible to re-assignate a different place for the different plot ? I mean : from left to right, the winner (HE-AAC) to loser (Real?). This may be useful, to see on which sample winner(s) fail(ed). Random position aren't useful in my opinion.

Yes, but that will take longer, since I'll have to redo all the excel plots.

I'll do it later tonight, since I must now leave to Uni.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
smok3
post Sep 22 2003, 15:43
Post #52


A/V Moderator


Group: Moderator
Posts: 1735
Joined: 30-April 02
From: Slovenia
Member No.: 1922



@rjamorim:

tnx for organizing and conducting the test.

flame verloren:
can't open http://audio.ciara.us/test/64test/comments...s/comments.html

B)


--------------------
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung
Go to the top of the page
+Quote Post
AstralStorm
post Sep 22 2003, 15:46
Post #53





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



Thank you rjamorim for making my results 'anon01'. biggrin.gif shifty.gif


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
rjamorim
post Sep 22 2003, 15:47
Post #54


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (smok3 @ Sep 22 2003, 11:43 AM)

OK, I changed comments.html to index.html.

So please use this link now:
http://audio.ciara.us/test/64test/comments/

Hopefully it'll work :B


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
rjamorim
post Sep 22 2003, 15:48
Post #55


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (AstralStorm @ Sep 22 2003, 11:46 AM)
Thank you rjamorim for making my results 'anon01'. biggrin.gif shifty.gif

Well, dude, you didn't specify if you wanted me to associate your results with your name... wink.gif

Now you're "just another number"


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
bond
post Sep 22 2003, 15:50
Post #56





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



good idea calling it low/high anchor (perhaps you should explain than on top of the page what these are, which encoder, settings and of course bitrate

QUOTE (guruboolez @ Sep 22 2003, 04:21 PM)
from left to right, the winner (HE-AAC) to loser (Real?). This may be useful, to see on which sample winner(s) fail(ed). Random position aren't useful in my opinion.

also nice
but i think it would be good if the order is the same in every plot, so perhaps the codecs should be ordered as they were judged in the overall results (so it is also very easy to see when a codec performed worse and when better)...

and one last optical thing:
if you do the plots again, plz write the codec names horizontally, perhaps use a smaller type size too because now the names are really too big imho (like half the size of the spreadsheet itself)


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
guruboolez
post Sep 22 2003, 15:51
Post #57





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Glad to know anon1's name. It's one of the rare listener that spoke about reverberation of vorbis encodings.
Go to the top of the page
+Quote Post
2Bdecided
post Sep 22 2003, 16:18
Post #58


ReplayGain developer


Group: Developer
Posts: 5175
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I was sad to be anon05 - I'll have to demand credit for my hard work next time - though I didn't work half as hard as you guys who did them all!

What about processed by rank rather than grade? Is that too much hard work?

Cheers,
David.
Go to the top of the page
+Quote Post
ff123
post Sep 22 2003, 16:31
Post #59


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (2Bdecided @ Sep 22 2003, 07:18 AM)
I was sad to be anon05 - I'll have to demand credit for my hard work next time - though I didn't work half as hard as you guys who did them all!

What about processed by rank rather than grade? Is that too much hard work?

Cheers,
David.

It should be easy to process by rank -- just run the raw data through the statistical tool again, this time using the friedman analysis.

ff123

Edit:
Both versions of abchr need to have a field for nickname (default nick is anonymous) and a checkbox to indicate whether the listener wants his comments associated with his nickname (just to be doubly sure about privacy); this would be written to a line in the results file.

This post has been edited by ff123: Sep 22 2003, 16:35
Go to the top of the page
+Quote Post
c_haese
post Sep 22 2003, 17:44
Post #60


Xiph.org Ogg Traffic editor


Group: Developer
Posts: 72
Joined: 7-July 02
Member No.: 2500



QUOTE (rjamorim @ Sep 22 2003, 09:47 AM)
OK, I changed comments.html to index.html.

So please use this link now:
http://audio.ciara.us/test/64test/comments/

Hopefully it'll work :B

It's not working for me. The page comes up, but none of the links on that page seem to work. Oh, and the title of that page says "AAC@128kbps test comments."

-Carsten
Go to the top of the page
+Quote Post
Niknak
post Sep 22 2003, 17:49
Post #61





Group: Members
Posts: 22
Joined: 22-September 03
Member No.: 8954



Can you give us the actual bitrates or filesizes with each codec and for each of the samples? Are there any significant differences?
Go to the top of the page
+Quote Post
de Mon
post Sep 22 2003, 18:42
Post #62





Group: Members
Posts: 474
Joined: 1-December 02
Member No.: 3940



I can't understand why didn't the best version of Vorbis used - GT3??? Why did realy hissy and not tuned version compared with quiet old and tuned products when there exists GT3? unsure.gif


--------------------
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech
Go to the top of the page
+Quote Post
Continuum
post Sep 22 2003, 18:45
Post #63





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



GT3 is only real tuned for high bitrates (>160 kbit). It would be no better at 64 kbit than the standard version.
Go to the top of the page
+Quote Post
sony666
post Sep 22 2003, 18:56
Post #64





Group: Members
Posts: 573
Joined: 22-February 02
Member No.: 1375



the mp3PRO result is interesting.. too bad the format never took off due to the licencing nightmare rolleyes.gif I see the same fate for AAC coming.. unless there will be a free and superior "LAME aac" encoder, mp3 is gonna stay (yes, except the divx ripping ppl)

thanks roberto e.a. for organizing this wink.gif

This post has been edited by sony666: Sep 22 2003, 19:00
Go to the top of the page
+Quote Post
JohnV
post Sep 22 2003, 19:10
Post #65





Group: Developer
Posts: 2797
Joined: 22-September 01
Member No.: 6



QUOTE (de Mon @ Sep 22 2003, 08:42 PM)
I can't understand why didn't the best version of Vorbis used - GT3??? Why did  realy hissy and not tuned version compared with quiet old and tuned products when there exists GT3? unsure.gif

The post 1.0 cvs version used includes actually some new tweakings for low bitrates, and as continuum said, it's gt3 which has nothing new for -q0.


--------------------
Juha Laaksonheimo
Go to the top of the page
+Quote Post
JohnV
post Sep 22 2003, 19:17
Post #66





Group: Developer
Posts: 2797
Joined: 22-September 01
Member No.: 6



QUOTE (sony666 @ Sep 22 2003, 08:56 PM)
the mp3PRO result is interesting.. too bad the format never took off due to the licencing  nightmare rolleyes.gif I see the same fate for AAC coming.. unless there will be a free and superior "LAME aac" encoder, mp3 is gonna stay (yes, except the divx ripping ppl)

1. mp3Pro is no standard -> AAC/AAC-HE is
2. mp3Pro is completely proprietary -> there are several different AAC implementations
3. FAAC is getting better and it's free. It might not be state of the art, but it's not that bad it used to be. I think it's very soon and partly already ready to challenge at least Lame MP3.

There's still no doubt that mp3 will be around for a looong time..


--------------------
Juha Laaksonheimo
Go to the top of the page
+Quote Post
AstralStorm
post Sep 22 2003, 19:22
Post #67





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



I can't read the comments from the site.

If I click any link on that page
after trying 20 times to connect and then getting always redirected
Mozilla says the site exceeds redirection limit.

This post has been edited by AstralStorm: Sep 22 2003, 20:16


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
de Mon
post Sep 22 2003, 21:54
Post #68





Group: Members
Posts: 474
Joined: 1-December 02
Member No.: 3940



Thanks you for your replies, Continuum and JohnV. I had not known about that info.


--------------------
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech
Go to the top of the page
+Quote Post
Dologan
post Sep 22 2003, 22:00
Post #69





Group: Members (Donating)
Posts: 478
Joined: 22-November 01
From: United Kingdom
Member No.: 519



Heh, I find it weird that I usually scored MP3Pro higher than HE-AAC. Those two (apart from Lame) were generally the only ones that needed careful listening to single out, but once I got hold of the artifacts I usually found HE-AAC's to be a little bit more annoying than those of MP3Pro, although there were times where HE-AAC was amazingly difficult to ABX for me, whereas that didn't happen with MP3Pro
Another deviation of mine from the average is that often WMA was usually almost at the level of FhG 64 in terms of annoyance, separated from QT and Real...

My overall: LAME > MP3Pro, HE-AAC > Ogg > Real = QT > WMA > FhG

~Dologan
Go to the top of the page
+Quote Post
ff123
post Sep 22 2003, 22:42
Post #70


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



Results have been slashdotted:

http://slashdot.org/article.pl?sid=03/09/2...tid=141&tid=188

Get your comments and rebuttals in while the thread's still hot.

ff123
Go to the top of the page
+Quote Post
AstralStorm
post Sep 22 2003, 23:03
Post #71





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



My results:
CODE
LAME     MP3Pro   HE-AAC   Vorbis   LC-AAC   Real     WMA      FHG
 4.07     3.39     3.13     2.67     2.22     2.05     1.77     1.55

LAME is better than MP3Pro, HE-AAC, Vorbis, LC-AAC, Real, WMA, FHG
MP3Pro is better than Vorbis, LC-AAC, Real, WMA, FHG
HE-AAC is better than LC-AAC, Real, WMA, FHG
Vorbis is better than WMA, FHG
LC-AAC is better than FHG


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
pseudoacoustic
post Sep 22 2003, 23:28
Post #72





Group: Members
Posts: 53
Joined: 28-August 03
Member No.: 8576



Great test Roberto. However, I also cannot access the comments page. Would it be too much to ask you to email them to me (the same email I sent you the results with)? Thanks.
Go to the top of the page
+Quote Post
rjamorim
post Sep 23 2003, 03:11
Post #73


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (ff123 @ Sep 22 2003, 06:42 PM)
Results have been slashdotted:

http://slashdot.org/article.pl?sid=03/09/2...tid=141&tid=188

Get your comments and rebuttals in while the thread's still hot.

ff123

QUOTE
"Hydrogenaudio has just wrapped up a listening test of various audio codecs at 64kbps."


That pisses me off. A lot.

Sure, it was "Hydrogenaudio" that stayed up till 5:30 AM sorting results, it was "Hydrogenaudio" that took shit on every issue that surfaced, it was "Hydrogenaudio" that screened 500+ results ditching the ones ranking the reference...

I'm not an asshole, but I think credit should be given where credit is due. Credits weren't given to ff123's test either, but at least it wasn't misrepresented as being conduced by HA.

sigh...

OK, rant off.


@pseudoacoustic: Check the 4th post in this same thread wink.gif
http://rarewares.hydrogenaudio.org/rja/comments.zip


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ff123
post Sep 23 2003, 04:17
Post #74


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



I always find stuff to reply to in the /. threads way too late. In the current thread there were a couple that got my goat (having to do with the legitimacy of subjective testing and the drawing of the confidence interval bars).

ff123
Go to the top of the page
+Quote Post
Dologan
post Sep 23 2003, 04:20
Post #75





Group: Members (Donating)
Posts: 478
Joined: 22-November 01
From: United Kingdom
Member No.: 519



QUOTE (rjamorim @ Sep 22 2003, 08:11 PM)
That pisses me off. A lot.

Sure, it was "Hydrogenaudio" that stayed up till 5:30 AM sorting results, it was "Hydrogenaudio" that took shit on every issue that surfaced, it was "Hydrogenaudio" that screened 500+ results ditching the ones ranking the reference...

I'm not an asshole, but I think credit should be given where credit is due. Credits weren't given to ff123's test either, but at least it wasn't misrepresented as being conduced by HA.

sigh...

OK, rant off.

Don't be sad, Roberto. Who needs Slashdot appreciation, anyway? We all love you here at HA.org, we all know you're da man. biggrin.gif

This post has been edited by dologan: Sep 23 2003, 04:21
Go to the top of the page
+Quote Post

5 Pages V  < 1 2 3 4 5 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st October 2014 - 21:40