IPB

Welcome Guest ( Log In | Register )

Probability of passing a sequencial ABX test, Split from "Mad Challenge - My Results"
KikeG
post Nov 10 2003, 14:39
Post #1


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



Edit : this discussion was started there.

I like viewing my ABX results during the test, and stopping when I have reached low enough p. That's what some people call a "sequential" ABX test. As Pio2001 says, for such a kind of test, getting a p = 5% is not enough pass the test. It seems (someone correct me if I'm wrong, please) that, for such a kind of sequential test, getting a p<1% is is enough for saying the the test has been passed.

This post has been edited by tigre: Nov 12 2003, 16:48
Go to the top of the page
+Quote Post
 
Start new topic
Replies
guruboolez
post Nov 10 2003, 18:25
Post #2





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Gabriel @ Nov 10 2003, 06:04 PM)
....
119 of 200, p = 0.004

...

(am I god?)

No, you missed 80 times the good answer, but pval tell you that there are few chances to guess. 7/8 is less significative than 60/100... In other words, if you're planning to perform a difficult ABX test, it's easier to obtain significant results by targetting 100 trials than 8.

Note that foobar2000 ABX component didn't compute pvalue after 20 trials.
Go to the top of the page
+Quote Post

Posts in this topic
- KikeG   Probability of passing a sequencial ABX test   Nov 10 2003, 14:39
- - guruboolez   QUOTE (KikeG @ Nov 10 2003, 02:39 PM)I like v...   Nov 10 2003, 14:55
- - KikeG   5% is valid only if either you have fixed in advan...   Nov 10 2003, 15:18
- - tigre   Is there a rule of thumb for tests without fixed n...   Nov 10 2003, 15:22
- - guruboolez   QUOTE (KikeG @ Nov 10 2003, 03:18 PM)5% is va...   Nov 10 2003, 15:25
- - KikeG   QUOTE (guruboolez @ Nov 10 2003, 03:25 PM)Sho...   Nov 10 2003, 15:51
- - KikeG   QUOTE (tigre @ Nov 10 2003, 03:22 PM)Is there...   Nov 10 2003, 15:53
- - tigre   QUOTE (KikeG @ Nov 10 2003, 04:53 PM)I don...   Nov 10 2003, 16:10
- - ScorLibran   QUOTE (tigre @ Nov 10 2003, 10:10 AM)I want t...   Nov 10 2003, 16:18
- - Lev   QUOTE Usually, a p <= 0.05 is considered a sign...   Nov 10 2003, 16:19
- - tigre   QUOTE (ScorLibran @ Nov 10 2003, 05:18 PM)QUO...   Nov 10 2003, 16:43
- - tigre   QUOTE (Lev @ Nov 10 2003, 05:19 PM)How is the...   Nov 10 2003, 16:44
- - guruboolez   QUOTE (KikeG @ Nov 10 2003, 03:51 PM)QUOTE (g...   Nov 10 2003, 17:33
- - AstralStorm   I'd very much appreciate an option (in ABC/HR ...   Nov 10 2003, 17:45
- - Gabriel   Based on the current discussion, I tryed an experi...   Nov 10 2003, 17:54
- - Gabriel   Still continuing the experiment: 54 of 85, p = 0...   Nov 10 2003, 17:58
- - guruboolez   QUOTE (Gabriel @ Nov 10 2003, 05:54 PM)Based ...   Nov 10 2003, 18:01
- - Gabriel   .... 119 of 200, p = 0.004 ... (am I god?)   Nov 10 2003, 18:04
- - ErikS   QUOTE (Gabriel @ Nov 10 2003, 06:04 PM).... 1...   Nov 10 2003, 18:15
- - guruboolez   QUOTE (Gabriel @ Nov 10 2003, 06:04 PM).... 1...   Nov 10 2003, 18:25
- - guruboolez   On ABX tests, if you did one error each three tria...   Nov 10 2003, 18:49
- - Moneo   QUOTE 119 of 200, p = 0.004 I wonder what does Win...   Nov 10 2003, 19:00
- - Continuum   QUOTE (Gabriel @ Nov 10 2003, 07:04 PM).... 1...   Nov 10 2003, 19:17
- - Continuum   QUOTE (guruboolez @ Nov 10 2003, 03:55 PM)Doe...   Nov 10 2003, 19:28
- - ff123   QUOTE (Gabriel @ Nov 10 2003, 08:58 AM)Still ...   Nov 10 2003, 19:31
- - schnofler   Just a few thoughts on this whole discussion: In g...   Nov 10 2003, 19:34
- - Moneo   The randomizer in WinABX does seem to be deficient...   Nov 10 2003, 19:36
- - guruboolez   QUOTE (schnofler @ Nov 10 2003, 07:34 PM)This...   Nov 10 2003, 20:14
- - tigre   QUOTE (Continuum @ Nov 10 2003, 08:28 PM)QUOT...   Nov 10 2003, 20:56
- - Continuum   QUOTE (tigre @ Nov 10 2003, 09:56 PM)Do you k...   Nov 10 2003, 21:06
- - Mac   QUOTE (Moneo @ Nov 10 2003, 06:36 PM)The rand...   Nov 10 2003, 21:18
- - AstralStorm   Heh, 116/200 is nearly 35% chance of missing accor...   Nov 10 2003, 21:28
- - Pio2001   The problem is obvious : in his random test, Gabri...   Nov 10 2003, 21:37
- - Continuum   QUOTE (Mac @ Nov 10 2003, 10:18 PM)QUOTE (Mon...   Nov 10 2003, 21:39
- - Continuum   QUOTE (AstralStorm @ Nov 10 2003, 10:28 PM)He...   Nov 10 2003, 21:42
- - Pio2001   QUOTE (Continuum @ Nov 10 2003, 09:42 PM)QUOT...   Nov 10 2003, 21:47
- - Mac   I was going on the standard deviation of a 202 tri...   Nov 10 2003, 21:57
- - schnofler   I think one little suggestion is necessary here: P...   Nov 10 2003, 22:00
- - Pio2001   You're right. I'd like to see a graph of ...   Nov 10 2003, 22:12
- - Moneo   QUOTE (Mac @ Nov 10 2003, 09:18 PM)Unless my ...   Nov 10 2003, 22:13
- - Continuum   QUOTE (Mac @ Nov 10 2003, 10:18 PM)both of th...   Nov 10 2003, 22:14
- - AstralStorm   Check with what probability you can get this resul...   Nov 10 2003, 22:30
- - schnofler   QUOTE (Moneo)Well, you could help debunking them b...   Nov 10 2003, 22:54
- - Mac   Erg, I mixed myself up a little When saying +/- ...   Nov 10 2003, 23:18
- - schnofler   QUOTE (Mac)Choosing all A: 98/200, p=63.8 Choosing...   Nov 10 2003, 23:20
- - Pio2001   Here's the graph for the Pval of my 200 answer...   Nov 10 2003, 23:20
- - Pio2001   Schnofler, could you post your program, of the log...   Nov 10 2003, 23:23
- - Gabriel   another test: 15 of 24, p = 0.154 16 of 25, p ...   Nov 10 2003, 23:34
- - schnofler   QUOTE (pio2001)Schnofler, could you post your prog...   Nov 10 2003, 23:42
- - Moneo   QUOTE (schnofler @ Nov 10 2003, 10:54 PM)OK, ...   Nov 10 2003, 23:46
- - Moneo   Anyway, I'd like to know what did KikeG use to...   Nov 11 2003, 00:00
- - Pio2001   Thanks, Schnofler, this is exactly what I wanted. ...   Nov 11 2003, 00:24
- - Pio2001   Note in the first graph, that if we only had the f...   Nov 11 2003, 00:28
- - schnofler   QUOTE (Pio2001)The second graph behaves exactly ho...   Nov 11 2003, 00:36
- - phong   There are many "weak" pseudorandom numbe...   Nov 11 2003, 02:27
- - ff123   QUOTE (phong @ Nov 10 2003, 05:27 PM)There ar...   Nov 11 2003, 02:58
- - Pio2001   Here's how the p value behaves, according to t...   Nov 11 2003, 05:05
- - KikeG   QUOTE (Continuum @ Nov 10 2003, 07:28 PM)But ...   Nov 11 2003, 09:33
- - KikeG   About WinABX PRNG, from v0.4 it also uses the Mers...   Nov 11 2003, 09:36
- - Moneo   QUOTE (KikeG @ Nov 11 2003, 09:36 AM)About Wi...   Nov 11 2003, 11:11
- - KikeG   QUOTE (Moneo @ Nov 11 2003, 11:11 AM)Could yo...   Nov 11 2003, 11:58
- - Moneo   QUOTE (KikeG @ Nov 11 2003, 11:58 AM)I tried ...   Nov 11 2003, 12:49
- - KikeG   QUOTE (Moneo @ Nov 11 2003, 12:49 PM)Maybe yo...   Nov 11 2003, 15:37
- - schnofler   Ok, I tested the new version, here you go: 1. 100...   Nov 11 2003, 16:10
- - tigre   the 3 following posts have been moved from Upsampl...   Nov 12 2003, 15:18
- - KikeG   QUOTE (tigre @ Nov 12 2003, 03:18 PM)I'm ...   Nov 12 2003, 15:35
- - tigre   Continuum's corrected p-values are only correc...   Nov 12 2003, 16:42
- - KikeG   QUOTE (tigre @ Nov 12 2003, 04:42 PM)Continuu...   Nov 12 2003, 17:15
- - Continuum   I'm not a statistics guru, but here is what I ...   Nov 12 2003, 18:51


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th September 2014 - 19:23