IPB

Welcome Guest ( Log In | Register )

listening tests help, for an audio codec I developed
legg
post Jun 18 2005, 19:51
Post #1





Group: Members
Posts: 175
Joined: 5-March 05
From: Morelia, Mexico
Member No.: 20386



How can I rate how good or bad will the audio codec be on average?

I'm thinking on doing at least 21 ABX trials on each file and if the subject managed to tell a difference and the guess probability is low I'd also take into account a 1-5 point scale to measure the perceived quality.

So far I have analized one subject, he did pretty well on most tests, except one, where the results where these:
15 out of 29, pval = 0.500

AFAIK, p<0.05 is good to be certain that he didn't know by chance, what about the rest?
How do I interpret that?, is the codec transparent for him at that test?
What are the ranges of the guess probabilities and how should I interpret them?

Thanks in advance.

This post has been edited by Luis G: Jun 18 2005, 19:52


--------------------
Home page: http://lc.fie.umich.mx/~legg/indexen.php
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Digga
post Jun 18 2005, 20:28
Post #2





Group: Members
Posts: 1047
Joined: 28-June 03
From: on the dock of the bay
Member No.: 7423



QUOTE (Luis G @ Jun 18 2005, 07:51 PM)
So far I have analized one subject, he did pretty well on most tests, except one, where the results where these:
15 out of 29, pval = 0.500

AFAIK, p<0.05 is good to be certain that he didn't know by chance, what about the rest?
How do I interpret that?, is the codec transparent for him at that test?
What are the ranges of the guess probabilities and how should I interpret them?
if you say he did pretty well except this one, the means that he could tell the difference right? for the codec it would be more complimenting (i.e. better) if he couldn't tell the difference which means it's transparent for him and he's guessing if x is a or b.

anyway, either alpha=0,01 or alpha=0,05 are generally chosen.
btw, you are not certain, but this is just (given, very low) probability that one is guessing, i.e. considered not guessing.
a pval of 0.5 would be a reference for guessing.

for further more in depth info, look here.

edit: damn am I slow...

This post has been edited by Digga: Jun 18 2005, 20:29


--------------------
Nothing but a Heartache - Since I found my Baby ;)
Go to the top of the page
+Quote Post
legg
post Jun 19 2005, 16:28
Post #3





Group: Members
Posts: 175
Joined: 5-March 05
From: Morelia, Mexico
Member No.: 20386



QUOTE (Digga @ Jun 18 2005, 01:28 PM)
if you say he did pretty well except this one, the means that he could tell the difference right? for the codec it would be more complimenting (i.e. better) if he couldn't tell the difference which means it's transparent for him and he's guessing if x is a or b.

anyway, either alpha=0,01 or alpha=0,05 are generally chosen.
btw, you are not certain, but this is just (given, very low) probability that one is guessing, i.e. considered not guessing.
a pval of 0.5 would be a reference for guessing.

for further more in depth info, look here.

edit: damn am I slow...
*



Yes, I have read that thread, but it doesn't mention how to deal with the results when p>0.05, which is this case. An interval like 0.05<p<0.25 how is to be dealt with? It certainly doesn't say much of the codec, and I would hardly classify it into transparency. IMO p>0.5 means transparency, but I wanted to check with you gurus about this.

I thought that anyone could rate the quality, but what if the subject merely rated the correct file by chance, I need more certainty, and I'm using ABX as an indicator of the trustworthy of the subject ratings. Is this correct?

Btw, I'm more interested in rating the subjective quality of the codec.

Thanks again.

This post has been edited by Luis G: Jun 19 2005, 16:46


--------------------
Home page: http://lc.fie.umich.mx/~legg/indexen.php
Go to the top of the page
+Quote Post

Posts in this topic


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 27th August 2014 - 15:11