IPB

Welcome Guest ( Log In | Register )

Minimum number of required ABX trials, Split from from topic ID: 92851
sauvage78
post Jan 12 2012, 00:41
Post #1





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



The minimal number of trials depends on how successfull you are.
How quickly you are sucessfull shows how confident in yourself you are.

The time & number of sucessfull trials are tied, you should never separate them when judging an ABX log.

With F2K ABX component, 8 sucessfull trials in a row (& if all successfull in a row, it usually means quick) trials is the minimum for me.

As soon as you begin to fail you can easyly increase to 10 or 12 to try to "erease" your failures.
In this case if you fail once or twice you can usually still get a signifiant result although it usually means the ABXing was hard, & by consequence longer as you begin to hesitate.

Usually if you begin to fail more than 3 times on 12 trials, it begins to be so hard & you have so much hesitation that it begins to take forever to ABX. At this stage I usually give up by myself & declare that I cannot ABX as in general it means I am not sure that the audio part I am focusing on actually contains any real artefact.

This post has been edited by sauvage78: Jan 12 2012, 00:42


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
 
Start new topic
Replies
sauvage78
post Jan 12 2012, 02:31
Post #2





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



Well I know this topic isn't about me wink.gif but my opinion on the topic is that ABXing is not pure statistics ... as soon as you select your sample & bitrate it is flawed statistics because you usually select the bitrate & sample in order to get failure from the start. Unlike a heads or tails coin flipping, nobody knows the average probability of failure\success of a target ABXing test because it depends both on how hard the ABX test is (samples\birate\encoders ...) & how good the listener is (ears\experience\patience ...). The coin is flawed, so applying pure math is good, but it has its real life limits.

So is there a minimal number of trials to identify an artefact ? for yourself the answer is NO, not really ... well usually 2 or 3 to be honest.
For yourself & yourself only, what matters is not the number of trials but the fact that you can identify the artefact. Identifying the artefact means that you know WHEN it happens in the sample & that you can DESCRIBE it. Knowing WHAT happens & WHEN it happens is what matters the most to me, because it means that you can ABX it for yourself 100% of time no matter the number of trials.

The number of trials is only usefull to convince others that you're not telling complete bullshits. This is why a minimum number of trials to get meaningfull statistical value is usefull.

Science means that you can repeat the experience. Once you know for yourself what you hear, the number of trials & how fast you can repeat your success is only usefull to convince others.
Obviously you need a higher number of trials to convince others than to convince yourself because they are lazy & won't double check your results.

So between 5 trials that is only good for yourself & 16 trials that is overkill for you, there is a real life "in between" which is statisticaly valid & usually it is between 8 & 12 depending on how sucessfull you are.

PS: Sorry if I was a little to extensive about myself...

Edit: Typo:expensive>extensive, as you can see saratoga my english is not perfect, so sometimes there are communication breakdowns. Sorry.

This post has been edited by sauvage78: Jan 12 2012, 02:51


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post

Posts in this topic
- sauvage78   Minimum number of required ABX trials   Jan 12 2012, 00:41
- - greynol   It is expected that you choose the number of trial...   Jan 12 2012, 00:44
- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 18:41) Th...   Jan 12 2012, 00:45
- - sauvage78   Well there is the theory & there is real life ...   Jan 12 2012, 00:56
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 18:56) To...   Jan 12 2012, 01:05
- - greynol   He should be conducting sets of 16 trials at first...   Jan 12 2012, 01:10
- - sauvage78   I never said I judged this test valid, I only gave...   Jan 12 2012, 01:13
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 19:13) I ...   Jan 12 2012, 01:17
|- - greynol   QUOTE (sauvage78 @ Jan 11 2012, 16:13) I ...   Jan 12 2012, 01:20
- - sauvage78   saratoga: Yes, I had the feeling that you were thi...   Jan 12 2012, 01:27
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 19:27) Ye...   Jan 12 2012, 01:32
|- - greynol   QUOTE (saratoga @ Jan 11 2012, 16:32) del...   Jan 12 2012, 01:42
- - sauvage78   Well I know this topic isn't about me but my...   Jan 12 2012, 02:31
- - greynol   The difference between you and the OP is that you ...   Jan 12 2012, 02:54
- - sauvage78   I don't even need a log anymore to trust /mnt ...   Jan 12 2012, 03:30
- - IgorC   A lot of discussion here but it won't change t...   Jan 12 2012, 03:55
|- - saratoga   QUOTE (IgorC @ Jan 11 2012, 21:55) A lot ...   Jan 12 2012, 04:06
- - sauvage78   Even if I think 5 trials is too low to convince ot...   Jan 12 2012, 04:01
|- - greynol   QUOTE (sauvage78 @ Jan 11 2012, 19:01) Ig...   Jan 12 2012, 04:13
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 22:01) Ev...   Jan 12 2012, 04:13
- - IgorC   I think I understand what sauvage78 wants to say. ...   Jan 12 2012, 04:46
|- - saratoga   QUOTE (IgorC @ Jan 11 2012, 22:46) It...   Jan 12 2012, 04:50
|- - IgorC   QUOTE (saratoga @ Jan 12 2012, 00:50) QUO...   Jan 12 2012, 04:57
- - sauvage78   QUOTE Are you claiming that I (or greynol) have no...   Jan 12 2012, 04:54
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 22:54) QU...   Jan 12 2012, 05:02
|- - IgorC   QUOTE (sauvage78 @ Jan 12 2012, 00:54) It...   Jan 12 2012, 05:03
|- - greynol   QUOTE (IgorC @ Jan 11 2012, 20:03) So it...   Jan 12 2012, 05:19
- - sauvage78   IgorC: I was more trying to say that if TOS8 is ve...   Jan 12 2012, 05:02
|- - IgorC   QUOTE (sauvage78 @ Jan 12 2012, 01:02) Ig...   Jan 12 2012, 05:06
- - nesf   From a complete newbie perspective: It could have ...   Jan 12 2012, 11:08
|- - saratoga   QUOTE (nesf @ Jan 12 2012, 05:08) From a ...   Jan 12 2012, 23:15
- - apodtele   Please read Fallacy of p-value. I just want to po...   Jan 12 2012, 17:09
|- - Porcus   QUOTE (apodtele @ Jan 12 2012, 17:09) Ple...   Jan 12 2012, 22:07
- - krabapple   It looks like we're groping towards a discussi...   Jan 12 2012, 18:00
- - nesf   A Dummies guide for doing some basic two sample an...   Jan 13 2012, 01:59


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 23rd October 2014 - 06:22