Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 64kbps public listening test (Read 61507 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

64kbps public listening test

Reply #50
Quote
Is it possible to re-assignate a different place for the different plot ? I mean : from left to right, the winner (HE-AAC) to loser (Real?). This may be useful, to see on which sample winner(s) fail(ed). Random position aren't useful in my opinion.

Yes, but that will take longer, since I'll have to redo all the excel plots.

I'll do it later tonight, since I must now leave to Uni.

64kbps public listening test

Reply #51
@rjamorim:

tnx for organizing and conducting the test.

flame verloren:
can't open http://audio.ciara.us/test/64test/comments...s/comments.html

B)
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

64kbps public listening test

Reply #52
Thank you rjamorim for making my results 'anon01'. 
ruxvilti'a


64kbps public listening test

Reply #54
Quote
Thank you rjamorim for making my results 'anon01'. 

Well, dude, you didn't specify if you wanted me to associate your results with your name...

Now you're "just another number"

64kbps public listening test

Reply #55
good idea calling it low/high anchor (perhaps you should explain than on top of the page what these are, which encoder, settings and of course bitrate

Quote
from left to right, the winner (HE-AAC) to loser (Real?). This may be useful, to see on which sample winner(s) fail(ed). Random position aren't useful in my opinion.

also nice
but i think it would be good if the order is the same in every plot, so perhaps the codecs should be ordered as they were judged in the overall results (so it is also very easy to see when a codec performed worse and when better)...

and one last optical thing:
if you do the plots again, plz write the codec names horizontally, perhaps use a smaller type size too because now the names are really too big imho (like half the size of the spreadsheet itself)
I know, that I know nothing (Socrates)

64kbps public listening test

Reply #56
Glad to know anon1's name. It's one of the rare listener that spoke about reverberation of vorbis encodings.

64kbps public listening test

Reply #57
I was sad to be anon05 - I'll have to demand credit for my hard work next time - though I didn't work half as hard as you guys who did them all!

What about processed by rank rather than grade? Is that too much hard work?

Cheers,
David.

64kbps public listening test

Reply #58
Quote
I was sad to be anon05 - I'll have to demand credit for my hard work next time - though I didn't work half as hard as you guys who did them all!

What about processed by rank rather than grade? Is that too much hard work?

Cheers,
David.

It should be easy to process by rank -- just run the raw data through the statistical tool again, this time using the friedman analysis.

ff123

Edit:
Both versions of abchr need to have a field for nickname (default nick is anonymous) and a checkbox to indicate whether the listener wants his comments associated with his nickname (just to be doubly sure about privacy); this would be written to a line in the results file.

64kbps public listening test

Reply #59
Quote
OK, I changed comments.html to index.html.

So please use this link now:
http://audio.ciara.us/test/64test/comments/

Hopefully it'll work :B

It's not working for me. The page comes up, but none of the links on that page seem to work. Oh, and the title of that page says "AAC@128kbps test comments."

-Carsten

64kbps public listening test

Reply #60
Can you give us the actual bitrates or filesizes with each codec and for each of the samples? Are there any significant differences?

64kbps public listening test

Reply #61
I can't understand why didn't the best version of Vorbis used - GT3??? Why did  realy hissy and not tuned version compared with quiet old and tuned products when there exists GT3?
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

64kbps public listening test

Reply #62
GT3 is only real tuned for high bitrates (>160 kbit). It would be no better at 64 kbit than the standard version.

64kbps public listening test

Reply #63
the mp3PRO result is interesting.. too bad the format never took off due to the licencing  nightmare  I see the same fate for AAC coming.. unless there will be a free and superior "LAME aac" encoder, mp3 is gonna stay (yes, except the divx ripping ppl)

thanks roberto e.a. for organizing this

64kbps public listening test

Reply #64
Quote
I can't understand why didn't the best version of Vorbis used - GT3??? Why did  realy hissy and not tuned version compared with quiet old and tuned products when there exists GT3?

The post 1.0 cvs version used includes actually some new tweakings for low bitrates, and as continuum said, it's gt3 which has nothing new for -q0.
Juha Laaksonheimo

64kbps public listening test

Reply #65
Quote
the mp3PRO result is interesting.. too bad the format never took off due to the licencing  nightmare  I see the same fate for AAC coming.. unless there will be a free and superior "LAME aac" encoder, mp3 is gonna stay (yes, except the divx ripping ppl)

1. mp3Pro is no standard -> AAC/AAC-HE is
2. mp3Pro is completely proprietary ->  there are several different AAC implementations
3. FAAC is getting better and it's free. It might not be state of the art, but it's not that bad it used to be. I think it's very soon and partly already ready to challenge at least Lame MP3.

There's still no doubt that mp3 will be around for a looong time..
Juha Laaksonheimo

64kbps public listening test

Reply #66
I can't read the comments from the site.

If I click any link on that page
after trying 20 times to connect and then getting always redirected
Mozilla says the site exceeds redirection limit.
ruxvilti'a

64kbps public listening test

Reply #67
Thanks you for your replies, Continuum and JohnV. I had not known about that info.
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

64kbps public listening test

Reply #68
Heh, I find it weird that I usually scored MP3Pro higher than HE-AAC. Those two (apart from Lame) were generally the only ones that needed careful listening to single out, but once I got hold of the artifacts I usually found HE-AAC's to be a little bit more annoying than those of MP3Pro, although there were times where HE-AAC was amazingly difficult to ABX for me, whereas that didn't happen with MP3Pro
Another deviation of mine from the average is that often WMA was usually almost at the level of FhG 64 in terms of annoyance, separated from QT and Real...

My overall: LAME > MP3Pro, HE-AAC > Ogg > Real = QT > WMA > FhG

~Dologan


64kbps public listening test

Reply #70
My results:
Code: [Select]
LAME     MP3Pro   HE-AAC   Vorbis   LC-AAC   Real     WMA      FHG
 4.07     3.39     3.13     2.67     2.22     2.05     1.77     1.55

LAME is better than MP3Pro, HE-AAC, Vorbis, LC-AAC, Real, WMA, FHG
MP3Pro is better than Vorbis, LC-AAC, Real, WMA, FHG
HE-AAC is better than LC-AAC, Real, WMA, FHG
Vorbis is better than WMA, FHG
LC-AAC is better than FHG
ruxvilti'a

64kbps public listening test

Reply #71
Great test Roberto. However, I also cannot access the comments page. Would it be too much to ask you to email them to me (the same email I sent you the results with)? Thanks.

64kbps public listening test

Reply #72
Quote
Results have been slashdotted:

http://slashdot.org/article.pl?sid=03/09/2...tid=141&tid=188

Get your comments and rebuttals in while the thread's still hot.

ff123

Quote
"Hydrogenaudio has just wrapped up a listening test of various audio codecs at 64kbps."


That pisses me off. A lot.

Sure, it was "Hydrogenaudio" that stayed up till 5:30 AM sorting results, it was "Hydrogenaudio" that took shit on every issue that surfaced, it was "Hydrogenaudio" that screened 500+ results ditching the ones ranking the reference...

I'm not an asshole, but I think credit should be given where credit is due. Credits weren't given to ff123's test either, but at least it wasn't misrepresented as being conduced by HA.

sigh...

OK, rant off.


@pseudoacoustic: Check the 4th post in this same thread
http://rarewares.hydrogenaudio.org/rja/comments.zip

64kbps public listening test

Reply #73
I always find stuff to reply to in the /. threads way too late.  In the current thread there were a couple that got my goat (having to do with the legitimacy of subjective testing and the drawing of the confidence interval bars).

ff123

 

64kbps public listening test

Reply #74
Quote
That pisses me off. A lot.

Sure, it was "Hydrogenaudio" that stayed up till 5:30 AM sorting results, it was "Hydrogenaudio" that took shit on every issue that surfaced, it was "Hydrogenaudio" that screened 500+ results ditching the ones ranking the reference...

I'm not an asshole, but I think credit should be given where credit is due. Credits weren't given to ff123's test either, but at least it wasn't misrepresented as being conduced by HA.

sigh...

OK, rant off.

Don't be sad, Roberto. Who needs Slashdot appreciation, anyway? We all love you here at HA.org, we all know you're da man.