IPB

Welcome Guest ( Log In | Register )

8 Pages V  « < 4 5 6 7 8 >  
Reply to this topicStart new topic
Multiformat Listening Test @ 128 kbps - FINISHED
Alex B
post Jan 16 2006, 11:54
Post #126





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Of course you are correct, guruboolez. My in-depth opinion is quite similar. Perhaps I simplified my answer a bit too much.

Actually, in the pretest discussion I tried to find some "low bitrate" human voice samples that would have been difficult for the encoders too, but with my limited experience in this I wasn't very successful. I posted a couple of jazz and opera voice samples (one of them was mono), but no one was impressed about them.


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 16 2006, 12:26
Post #127


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



QUOTE (Serge Smirnoff @ Jan 16 2006, 07:05 AM)
If my memory serves me, Ivan promised to publish results of PEAQ analyses of encoded sound samples from the listening test. May we hope to see them?
*


Yes,

Here they are (XLS file) - files were processed with the reference commercial implementation of the Advanced PEAQ analysis (ITU BS.1387 / Advanced PEAQ) with automatic delay and gain compensation (pretty much default options)

So - Advanced PEAQ rated encoders like this:

iTunes: 4.46
Nero: 4.49
LAME: 4.33
Vorbis: 4.45
WMA: 4.44
Shine: 2.48

Here are raw results, starting from the last sample, down to the first:

CODE
    iTunes    Nero    LAME    Vorbis    WMA    Shine
Sample 18    -0.291379    -0.22438    -0.551136    -0.410874    -0.464171    -2.89086
Sample 17    -0.485661    -0.625404    -0.558785    -0.477524    -0.517271    -3.32223
Sample 16    -0.562372    -0.474273    -0.82978    -0.430512    -0.62892    -1.97767
Sample 15    -0.536917    -0.397598    -0.684565    -0.604991    -0.757444    -3.09543
Sample 14    -0.55296    -0.601416    -0.639036    -0.59632    -0.557496    -2.68244
Sample 13    -0.523003    -0.533149    -0.504188    -0.646469    -0.395212    -2.82543
Sample 12    -0.584267    -0.626797    -0.710563    -0.615299    -0.559485    -1.73998
Sample 11    -0.539979    -0.455684    -0.604449    -0.470218    -0.518138    -1.71468
Sample 10    -0.414742    -0.469117    -0.701445    -0.31328    -0.710211    -2.08605
Sample 09    -0.698941    -0.720779    -0.729232    -0.743117    -0.515512    -3.49285
Sample 08    -0.595305    -0.66342    -0.615311    -0.57183    -0.566153    -3.1158
Sample 07    -0.430192    -0.422099    -0.765232    -0.668022    -0.545455    -2.10136
Sample 06    -0.763366    -0.405315    -0.626802    -0.467821    -0.594331    -1.00229
Sample 05    -0.648616    -0.682204    -0.512671    -0.625941    -0.458714    -3.0605
Sample 04    -0.573599    -0.545224    -0.625719    -0.605796    -0.493474    -3.22439
Sample 03    -0.454201    -0.418986    -1.21393    -0.531274    -0.833721    -1.35914
Sample 02    -0.497593    -0.431844    -0.524538    -0.578111    -0.453721    -2.72985
Sample 01    -0.583718    -0.501613    -0.654598    -0.565685    -0.567475    -2.96624
ANOVA    -0.54    -0.51    -0.67    -0.55    -0.56    -2.52
Worst Item    -0.763366    -0.720779    -1.21393    -0.743117    -0.833721    -3.49285
Best Item    -0.291379    -0.22438    -0.504188    -0.31328    -0.395212    -1.00229


This post has been edited by Ivan Dimkovic: Jan 16 2006, 12:30
Attached File(s)
Attached File  multiformat_128kbps_listeningtest_07Dec05.rar ( 2.94K ) Number of downloads: 347
 
Go to the top of the page
+Quote Post
Sebastian Mares
post Jan 16 2006, 12:35
Post #128





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (kwanbis @ Jan 16 2006, 12:12 AM)
QUOTE (Sebastian Mares @ Jan 15 2006, 10:43 PM)
Sorry, no.
You can do it yourself and state that it's your plot - I have no problem with that. I am not responsible for the way people post that image and what they do and don't do. I doubt that image is going to appear on a forum out of the sudden and without any comments - people who post the plot can also create a link below or above. The encoder version is more than enough in the graphics.
Discussion about this is over.
*

ok then, can you post the xls/openoffice/whatever you used?
*



http://maresweb.de/listening-tests/mf-128-1/pandts.xls


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
guruboolez
post Jan 16 2006, 12:40
Post #129





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
Go to the top of the page
+Quote Post
Sebastian Mares
post Jan 16 2006, 12:40
Post #130





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (rjamorim @ Jan 16 2006, 12:30 AM)
Very big congratulations to Sebastian for managing to produce interesting results after all the hardships he had to go through to discuss his test and later conduct it.

And here's to your upcoming tests! May the next test (64kbps?) be as successfull (but less stressful!) than this one. I know for sure I can hardly wait for it... smile.gif
*


Ivan is doing some pre-tests to decide which Nero encoder to use (AFAIK). I would love to conduct the next low-bitrate test (maybe with Shade[ST] since he's interested in running a test, too).

QUOTE (Caroliano @ Jan 16 2006, 02:11 AM)
How I can decrypt the results with the key.key file?
*


ABC/HR has an option like Tools --> Process results or something (sorry, the school PC I am currently using does not have Java installed).


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Gabriel
post Jan 16 2006, 12:42
Post #131


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
So - Advanced PEAQ rated encoders like this:

As you have access to both standard and advanced PEAQ, does the advanced version really provide more real life correlation?
Go to the top of the page
+Quote Post
Sebastian Mares
post Jan 16 2006, 12:43
Post #132





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



I think the PEAQ analysis is not reliable since it is based on psy-models like most lossy encoders. If an encoder has the same (or at least very similar) psy-model as the tool used for testing, the PEAQ analysis is going to rate the respective codec as excellent, even though the psy-model of the tested encoder might not achieve good results. Of course, PEAQ uses a very highly tuned model, but still, it is not as reliable as real-world testing with human beings.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Gabriel
post Jan 16 2006, 12:49
Post #133


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
If an encoder has the same (or at least very similar) psy-model as the tool used for testing, the PEAQ analysis is going to rate the respective codec as excellent, even though the psy-model of the tested encoder might not achieve good results.

Of course, and it seems clear that Ivan is using advanced PEAQ often, based on the PEAQ results of Nero.
However, even with the same psymodel, in a bitrate limited scenario, there is the bit allocation part that you will impact results.
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 16 2006, 13:00
Post #134


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



QUOTE
Gabriel
As you have access to both standard and advanced PEAQ, does the advanced version really provide more real life correlation?


I would have to re-do the test with the Basic PEAQ, but today I have absolutely no time for that - as Basic PEAQ is publically available (AFsp package / PQEvalAudio) someone could also do it - if not, I'll try over the week.

As for the Advanced PEAQ and correlation - I cannot comment this, as I have not performed too much correlation tests, but judging from the ITU BS.1387 papers on AES conferences, they indeed achieved higher correlation with the real-world listening test data by usign Advanced PEAQ, than with the Basic (FFT) model. Correlation was one of the factors for forming the "basic" and "advanced" PEAQ out of the so-called "toolbox" (set of algorithmic tools).

QUOTE
Sebastian MaresI think the PEAQ analysis is not reliable since it is based on psy-models like most lossy encoders. If an encoder has the same (or at least very similar) psy-model as the tool used for testing, the PEAQ analysis is going to rate the respective codec as excellent, even though the psy-model of the tested encoder might not achieve good results. Of course, PEAQ uses a very highly tuned model, but still, it is not as reliable as real-world testing with human beings.


Hmm - I think the purpose of PEAQ is not to replace the subjective listening tests - but to assist developers / engineers in situations where subjective tests would be tpp slow (even impossible) and/or too expensive - for example, where a lot of tests need to be made in the short time for many tools, doing subjective tests for each setting/tool/combination would be impossible.

Therefore PEAQ can only be "more or less" correlated to the real-world data - by no means they could replace real human being and its judgment, especially a lot of high trained humans in a properly done test smile.gif

As for the psymodel - as Gabriel said, there is much more in the codec that generates noise than just the psymodel's estimation - bit allocation over time and frequency, quantization properties, stereo coding, lossless (noiseless) coding step, etc... So, tweaking the psymodel just to match PEAQ values probably won't lead to the best ODG result.

This post has been edited by Ivan Dimkovic: Jan 16 2006, 13:12
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 16 2006, 13:13
Post #135


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



QUOTE (guruboolez @ Jan 16 2006, 11:40 AM)
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
*


Hmm no idea at all sad.gif But I also have few samples exibiting similar behavior, but for AAC.
Go to the top of the page
+Quote Post
loophole
post Jan 16 2006, 13:17
Post #136





Group: Members
Posts: 273
Joined: 18-June 03
Member No.: 7254



Just quickly going back to AAC players - besides the ones already mentioned there is http://www.panasonic-europe.com/news_read.aspx?id=2091 this panasonic one (and an older model), the Sony Walkman W800 (and K750), and the PSP. If Sony are allowing AAC playback on their phones and PSP i wouldn't be suprised to see support on other stuff like their network walkmans or Hi-MD later on. I think the Hi-MD devices already playback mp3 natively (not that you'd ever want to buy one).
Go to the top of the page
+Quote Post
kwanbis
post Jan 16 2006, 13:54
Post #137





Group: Developer (Donating)
Posts: 2390
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (Sebastian Mares @ Jan 16 2006, 11:35 AM)

thanks


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
Alex B
post Jan 16 2006, 14:16
Post #138





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (guruboolez @ Jan 16 2006, 01:40 PM)
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
*

At first, I thought that sample 3 was similarly odd too:

QUOTE
iTunes Nero LAME Vorbis WMA Shine

Sample 03 -0.454201 -0.418986 -1.21393 -0.531274 -0.833721 -1.35914

but then I checked my test report and found out that I too ranked LAME and Shine similarly poorer than the others (which I couldn't differentiate):

CODE
ABC/HR for Java, Version 0.5b, 13 January 2006
Testname: Carbonelli

Tester:

1L = Sample03\Carbonelli_3.wav
2L = Sample03\Carbonelli_1.wav
3R = Sample03\Carbonelli_2.wav
4L = Sample03\Carbonelli_6.wav
5L = Sample03\Carbonelli_4.wav
6L = Sample03\Carbonelli_5.wav

---------------------------------------
General Comments:
---------------------------------------
3R File: Sample03\Carbonelli_2.wav
3R Rating: 3.8
3R Comment: Artifact in the very first long note
---------------------------------------
5L File: Sample03\Carbonelli_4.wav
5L Rating: 3.8
5L Comment: Sounds a little "unstable". Probably the low anchor.

Carbonelli is the only sample where low anchor is not obvious.
---------------------------------------

ABX Results:
Original vs Sample03\Carbonelli_4.wav
11 out of 12, pval = 0.0030
Original vs Sample03\Carbonelli_2.wav
17 out of 17, pval < 0.001


---- Detailed ABX results ----
Original vs Sample03\Carbonelli_4.wav
Playback Range: 14.227 to 15.868
3:12:42 AM f 0/1 pval = 1.0
3:12:45 AM p 1/2 pval = 0.75
3:12:48 AM p 2/3 pval = 0.5
3:12:51 AM p 3/4 pval = 0.312
3:12:54 AM p 4/5 pval = 0.187
3:12:57 AM p 5/6 pval = 0.109
3:12:59 AM p 6/7 pval = 0.062
3:13:02 AM p 7/8 pval = 0.035
3:13:05 AM p 8/9 pval = 0.019
3:13:08 AM p 9/10 pval = 0.01
3:13:11 AM p 10/11 pval = 0.0050
3:13:15 AM p 11/12 pval = 0.0030

Original vs Sample03\Carbonelli_2.wav
Playback Range: 00.000 to 01.329
3:20:40 AM p 1/1 pval = 0.5
3:20:48 AM p 2/2 pval = 0.25
3:20:51 AM p 3/3 pval = 0.125
3:20:54 AM p 4/4 pval = 0.062
3:21:05 AM p 5/5 pval = 0.031
3:21:32 AM p 6/6 pval = 0.015
3:21:35 AM p 7/7 pval = 0.0070
3:21:37 AM p 8/8 pval = 0.0030
3:21:42 AM p 9/9 pval = 0.0010
3:21:46 AM p 10/10 pval < 0.001
3:21:54 AM p 11/11 pval < 0.001
3:21:57 AM p 12/12 pval < 0.001
3:22:01 AM p 13/13 pval < 0.001
3:22:03 AM p 14/14 pval < 0.001
3:22:06 AM p 15/15 pval < 0.001
3:22:09 AM p 16/16 pval < 0.001
3:22:12 AM p 17/17 pval < 0.001


As a comparison, guruboolez's results were these:

CODE
ABC/HR for Java, Version 0.5b, 07 décembre 2005
Testname: Carbonelli

Tester: guruboolez

1L = Sample03\Carbonelli_5.wav
2L = Sample03\Carbonelli_1.wav
3L = Sample03\Carbonelli_2.wav
4L = Sample03\Carbonelli_6.wav
5R = Sample03\Carbonelli_4.wav
6R = Sample03\Carbonelli_3.wav

---------------------------------------
General Comments:
---------------------------------------
1L File: Sample03\Carbonelli_5.wav
1L Rating: 4.8
1L Comment: very very small ringing (I'm surprised myself by the ABX score I get)
---------------------------------------
2L File: Sample03\Carbonelli_1.wav
2L Rating: 3.8
2L Comment: little ringing
---------------------------------------
3L File: Sample03\Carbonelli_2.wav
3L Rating: 2.0
3L Comment: tremolo effect; very minor kind of warbling also
---------------------------------------
4L File: Sample03\Carbonelli_6.wav
4L Rating: 4.3
4L Comment: minor distortion (unsure -> need ABXing)
---------------------------------------
5R File: Sample03\Carbonelli_4.wav
5R Rating: 1.3
5R Comment: severe ringing
---------------------------------------
6R File: Sample03\Carbonelli_3.wav
6R Rating: 4.3
6R Comment: same kind of problem than 4L
---------------------------------------

ABX Results:
Original vs Sample03\Carbonelli_6.wav
8 out of 8, pval = 0.0030
Original vs Sample03\Carbonelli_5.wav
7 out of 8, pval = 0.035
Original vs Sample03\Carbonelli_3.wav
7 out of 8, pval = 0.035


---- Detailed ABX results ----
Original vs Sample03\Carbonelli_6.wav
Playback Range: 00.000 to 02.695
8:15:57 PM p 1/1 pval = 0.5
8:16:01 PM p 2/2 pval = 0.25
8:16:07 PM p 3/3 pval = 0.125
8:16:09 PM p 4/4 pval = 0.062
8:16:12 PM p 5/5 pval = 0.031
8:16:16 PM p 6/6 pval = 0.015
8:16:21 PM p 7/7 pval = 0.0070
8:16:25 PM p 8/8 pval = 0.0030

Original vs Sample03\Carbonelli_5.wav
Playback Range: 00.000 to 02.695
8:18:32 PM p 1/1 pval = 0.5
8:18:36 PM p 2/2 pval = 0.25
8:18:41 PM p 3/3 pval = 0.125
8:18:51 PM p 4/4 pval = 0.062
8:18:55 PM p 5/5 pval = 0.031
8:18:59 PM p 6/6 pval = 0.015
8:19:03 PM p 7/7 pval = 0.0070
8:19:16 PM f 7/8 pval = 0.035

Original vs Sample03\Carbonelli_3.wav
Playback Range: 00.000 to 02.695
8:17:05 PM p 1/1 pval = 0.5
8:17:09 PM p 2/2 pval = 0.25
8:17:26 PM p 3/3 pval = 0.125
8:17:31 PM p 4/4 pval = 0.062
8:17:34 PM p 5/5 pval = 0.031
8:17:38 PM f 5/6 pval = 0.109
8:17:45 PM p 6/7 pval = 0.062
8:17:49 PM p 7/8 pval = 0.035


Here's a summary:

PEAQ _ Alex B _ guruboolez

iTunes: -0.45 (~4.5) _ 5.0 _ 3.8
Lame: -1.2 (~3.8) _ 3.8 _ 2.0
Nero: -0.42 (~4.6) _ 5.0 _ 4.3
Shine: -1.36 (~3.6) _ 3.8 _ 1.3
Vorbis: -0.53 (~4.5) _ 5.0 _ 4.8
WMA: -0.83 (~4.2) _ 5.0 _ 4.3

All three testers found LAME and Shine clearly worse than the others with this sample. Advanced PEAQ found WMA Pro slightly worse than the AAC codecs or Vorbis. Guru found that iTunes had a bit more problems than Nero or WMA and ranked Vorbis to be the best.

EDIT

The overall results for this sample:



Artist: Giovanni Stefano Carbonelli
Title: Sonata Settima In La Minore
Genre: Baroque Chamber Music
Sumbitted by: guruboolez

Nero score: 4.90

iTunes, AoTuV and WMA Professional are tied on first place. LAME and Shine are tied on last place.

EDIT 2: added PEAQ resuts in scale 0.0 - 5.0

This post has been edited by Alex B: Jan 16 2006, 15:05


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
sehested
post Jan 16 2006, 14:50
Post #139





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (guruboolez @ Jan 16 2006, 03:40 AM)
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
*
Several individuals have mistaken LAME for the low anchor on this sample. Could be by coincidence or... LAME has an issue with SongForGuy.
Go to the top of the page
+Quote Post
CoRoNe
post Jan 16 2006, 14:51
Post #140





Group: Members
Posts: 187
Joined: 31-May 05
From: Netherlands
Member No.: 22417



Perhaps this question has been asked many times, but I'd like to know what it is what makes this aoTuVb4.51 version so extremely good compared to the libvorbis 1.12 version? I even hear "the resurrection of Vorbis" is due to aoTuV!
If the aoTuVb4.51 has a score of 4,79 in this test, what average score would the libvorbis 1.12 have compared to the aoTuV and all the others!??


--------------------
DC-Bass Source Mod: http://reino.degeelebosch.nl
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 16 2006, 14:55
Post #141


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



One note about Advanced PEAQ, and the ratings - its neural network was trained by the large listening test database up to 1997.

In addition, in its tool set, PEAQ itself does not take into account stereo effects too much, and it might be the reason of the difference in real rating between, say, Vorbis - which is using lossy stereo above (if I am not mistaken) 10 kHZ and PEAQ estimation.

It could be that PEAQ's model treats distortion introduced by the lossy stereo as much more relevant as it is to the most of the human listeners due to a HF stereo masking not taken into account.

What would be interesting, if someone starts the project of extending the PEAQ by more elaborate binaural hearing models, as well as training on the more recent listening tests.

This post has been edited by Ivan Dimkovic: Jan 16 2006, 14:56
Go to the top of the page
+Quote Post
sehested
post Jan 16 2006, 18:03
Post #142





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (Sebastian Mares @ Jan 15 2006, 06:51 AM)
QUOTE (Garf @ Jan 15 2006, 03:29 PM)
Let me rephrase: How many of the grades given were 5.0? And how much if you re-add the ranked references as meaning that codec got a 5.0 for that sample?

So, there's 403 valid test results times 5 codecs (Shine doesn't count), or about 2015 grades. How many of those are 5.0, i.e. perfectly transparent?
*


Geez, no idea. That would take too much time - time that I don't have right now. I could send you all results if you really want to do it yourself.
*


@Sebastian: I like to have a go at it. smile.gif Just tell me how to obtain the files.

Edit: Speliing

This post has been edited by sehested: Jan 16 2006, 18:06
Go to the top of the page
+Quote Post
Sebastian Mares
post Jan 16 2006, 19:58
Post #143





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (sehested @ Jan 16 2006, 06:03 PM)
QUOTE (Sebastian Mares @ Jan 15 2006, 06:51 AM)
QUOTE (Garf @ Jan 15 2006, 03:29 PM)
Let me rephrase: How many of the grades given were 5.0? And how much if you re-add the ranked references as meaning that codec got a 5.0 for that sample?

So, there's 403 valid test results times 5 codecs (Shine doesn't count), or about 2015 grades. How many of those are 5.0, i.e. perfectly transparent?
*


Geez, no idea. That would take too much time - time that I don't have right now. I could send you all results if you really want to do it yourself.
*


@Sebastian: I like to have a go at it. smile.gif Just tell me how to obtain the files.

Edit: Speliing
*



Damn, while re-sorting all files for you now, I noticed a mistake caused by an improper naming of a result. Sample 8 (eric_clapton) has a result called "anon16-result08" which is in fact for Elizabeth. "anon16-result07" is identical to "anon16-result08". I am going to re-screen the results for sample 8 and then re-do the final plots. Sorry folks! I am sure all other results are in order.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Sebastian Mares
post Jan 16 2006, 20:18
Post #144





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



The plot for sample 8 was updated - only minor changes, nothing that would change the overall results.

Still working on the final plots now. The only difference is the changed Tukey's HSD value (change of 0.001) and the lower ranking of Shine only.

Edit: All problems solved now. As stated, only the non-zoomed plot was affected (and there, only Shine is affected - it lost 0.01 points).

This post has been edited by Sebastian Mares: Jan 16 2006, 20:43


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
QuantumKnot
post Jan 17 2006, 01:13
Post #145





Group: Developer
Posts: 1245
Joined: 16-December 02
From: Australia
Member No.: 4097



QUOTE (Nayru @ Jan 16 2006, 04:43 PM)
Vorbis did not finish 'at the front'.  The difference between AoTuV and iTunes AAC is within the margin of error and is not significant.
*


I didn't say it was alone at the front, did I?
Go to the top of the page
+Quote Post
kwanbis
post Jan 17 2006, 02:17
Post #146





Group: Developer (Donating)
Posts: 2390
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (Sebastian Mares @ Jan 16 2006, 07:18 PM)
Edit: All problems solved now. As stated, only the non-zoomed plot was affected (and there, only Shine is affected - it lost 0.01 points).

updated xls?


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
Caroliano
post Jan 17 2006, 03:58
Post #147





Group: Members
Posts: 67
Joined: 21-December 05
Member No.: 26559



QUOTE
ABC/HR has an option like Tools --> Process results or something (sorry, the school PC I am currently using does not have Java installed).

Thanks, it worked. It shoud be explaned and linked in somewere, because with search is dificult to find that.

The decoded sample12 result that I don't send:
CODE
ABC/HR for Java, Version 0.5b, 13 Janeiro 2006
Testname: MysteriousTimes

Tester: Caroliano

1R = Sample12\MysteriousTimes_5.wav
2R = Sample12\MysteriousTimes_6.wav
3L = Sample12\MysteriousTimes_2.wav
4R = Sample12\MysteriousTimes_3.wav
5L = Sample12\MysteriousTimes_4.wav
6R = Sample12\MysteriousTimes_1.wav

---------------------------------------
General Comments:
---------------------------------------
1R File: Sample12\MysteriousTimes_5.wav
1R Rating: 4.0
1R Comment:
---------------------------------------
2R File: Sample12\MysteriousTimes_6.wav
2R Rating: 4.2
2R Comment:
---------------------------------------
3L File: Sample12\MysteriousTimes_2.wav
3L Rating: 4.2
3L Comment:
---------------------------------------
5L File: Sample12\MysteriousTimes_4.wav
5L Rating: 2.5
5L Comment: low-anchor
---------------------------------------

ABX Results:

Again I rated vorbis as the worst sounding. I must be sensible for Vorbis artfacts...

PS: I was anon26, but is ok to display my name. I only forgot to put my name in them...
Go to the top of the page
+Quote Post
Sebastian Mares
post Jan 17 2006, 08:00
Post #148





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (kwanbis @ Jan 17 2006, 02:17 AM)
QUOTE (Sebastian Mares @ Jan 16 2006, 07:18 PM)
Edit: All problems solved now. As stated, only the non-zoomed plot was affected (and there, only Shine is affected - it lost 0.01 points).

updated xls?
*



Everything - plots, the results page, the results RAR... smile.gif

This post has been edited by Sebastian Mares: Jan 17 2006, 08:01


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
sehested
post Jan 17 2006, 19:05
Post #149





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (Garf @ Jan 15 2006, 06:29 AM)
How many of the grades given were 5.0? And how much if you re-add the ranked references as meaning that codec got a 5.0 for that sample?

So, there's 403 valid test results times 5 codecs (Shine doesn't count), or about 2015 grades. How many of those are 5.0, i.e. perfectly transparent?
*

@Sebastian: Thanks for the test results. wink.gif

I managed to proces the results and produce this table:
CODE
Ranked refs      24     14     18      6     19     25
5.0's           304    260    299     36    313    302
5.0's %          75%    65%    74%     9%    78%    75%
4.0 and above   361    334    358     60    375    355
4.0 and above %  90%    83%    89%    15%    93%    88%
Go to the top of the page
+Quote Post
kwanbis
post Jan 17 2006, 19:24
Post #150





Group: Developer (Donating)
Posts: 2390
Joined: 28-June 02
From: Argentina
Member No.: 2425



[quote=Sebastian Mares,Jan 17 2006, 07:00 AM]
updated xls?[/quote]

Everything - plots, the results page, the results RAR... smile.gif[/quote]
i mean, do you have an updated xls to download?


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post

8 Pages V  « < 4 5 6 7 8 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 29th December 2014 - 13:08