IPB

Welcome Guest ( Log In | Register )

listening tests help, for an audio codec I developed
legg
post Jun 18 2005, 19:51
Post #1





Group: Members
Posts: 175
Joined: 5-March 05
From: Morelia, Mexico
Member No.: 20386



How can I rate how good or bad will the audio codec be on average?

I'm thinking on doing at least 21 ABX trials on each file and if the subject managed to tell a difference and the guess probability is low I'd also take into account a 1-5 point scale to measure the perceived quality.

So far I have analized one subject, he did pretty well on most tests, except one, where the results where these:
15 out of 29, pval = 0.500

AFAIK, p<0.05 is good to be certain that he didn't know by chance, what about the rest?
How do I interpret that?, is the codec transparent for him at that test?
What are the ranges of the guess probabilities and how should I interpret them?

Thanks in advance.

This post has been edited by Luis G: Jun 18 2005, 19:52


--------------------
Home page: http://lc.fie.umich.mx/~legg/indexen.php
Go to the top of the page
+Quote Post
 
Start new topic
Replies
ff123
post Jun 19 2005, 17:34
Post #2


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



If it were me, I'd just choose a whole bunch of different samples (say 30), and then rate each one against the reference using abc-hr. Then I'd plug the results into a statistical calculator (http://ff123.net/friedman/stats.html) to determine first if you found a significant difference from the reference and second how much that difference is. No ABX'ing is involved, plus you get a better indicator of codec quality by sampling a lot of different music.

BTW, I would also keep the samples where you rate the reference, rather than throw these cases out. So if you make some mistakes, the reference will average something less than 5.0.

ff123
Go to the top of the page
+Quote Post
guruboolez
post Jun 19 2005, 18:55
Post #3





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (ff123 @ Jun 19 2005, 05:34 PM)
If it were me, I'd just choose a whole bunch of different samples (say 30), and then rate each one against the reference using abc-hr.
*


I'm going off-topic, but I've an important question.
I'm trying to build a complete set of classical music sample, in order to replace the usual suit of 15 samples I'm using now for 18 months. My purpose is to obtain 100 samples, including many instruments, solo, chamber, orchestral, lyrical, noisy or not noisy, quiet and loud, etc... But I'm realizing that making ABX comparisons with so many samples would be a Herculean task.
What would be the best thing in your opinion:
- 100 samples rated in ABC/HR without ABX
- 15...20 samples rated in ABC/HR + ABX confirmation?
Go to the top of the page
+Quote Post

Posts in this topic


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 28th December 2014 - 02:07