Calculating the PValue 
Calculating the PValue 
Oct 30 2003, 01:38
Post
#1


Group: Members Posts: 391 Joined: 24December 02 From: Eugene, OR Member No.: 4224 
Does anyone want to give me the formula? Thanks.
 http://www.pkulak.com



Oct 30 2003, 04:23
Post
#2


Group: Members Posts: 346 Joined: 7July 03 From: 15 & Ryan Member No.: 7619 
I was looking for the same thing myself... And I found it. I don't think I can insert mathML here, and I think code is a much better explanation than stupid mathy symbols, sere's some code I wrote minus some comments. It could be written to be much faster, but it is plenty fast enough for the number of trials we'd be talking about. Can you read Python?
CODE def factorial(x): if x < 0: raise ValueError("Factorials for numbers less than zero are undefined") if x <= 1: return 1 return reduce(operator.mul, xrange(2, x+1)) def binomial(n, k, p=0.5): if k > n: raise ValueError("Number of occurances (%d) exceeds number of trials (%d)." % (k, n)) q = 1.0  p return pow(p, k) * pow(q, nk) * factorial(n) / (factorial(k) * factorial(nk)) def abxBinomial(trials, correct): if correct > trials: raise ValueError("Number of correct ABX trials (%d) exceeds total number of trials (%d)." % (correct, trials) return reduce(operator.add, map(lambda k: binomial(trials, k), range(correct, trials+1))) Quick explanation: factorial() does exactly what its name says. It computes the factorial of a number. binomial() uses the binomial distribution function to compute the probability that there would be exactly k occurances of some event of probability p occuring in n trials. abxBinomial() computes the ABX pval you want. It takes the sum of the binomial function for correct, correct+1 ... num_trials. This method is fine for use on a computer/calculator, but if you're working it out by hand there are formulas which approximate the value more quickly (if less accurately). Also, be aware that it overflows doubles if you take 170+ trials.  I am *expanding!* It is so much *squishy* to *smell* you! *Campers* are the best! I have *anticipation* and then what? Better parties in *the middle* for sure.
http://www.phong.org/ 


Oct 30 2003, 05:32
Post
#3


Group: Members Posts: 210 Joined: 16June 03 Member No.: 7218 
QUOTE (phong @ Oct 30 2003, 04:23 AM) whatever I admint I am not an expert, but the socalled Pvalue probably refers to the Greek letter Rho, which looks pretty much like a P. Rho stands for probability. It is quite easy to calculate the value. Say you have a normal 6 edged dice. The probability (Rho) of getting a 6 in the first try is 1/6 ~ 16,7%. The probability (Rho) of getting 6 in the fist try and 2 in the second try is 1/6 * 1/6 ~ 2,7%. When you are doing simple ABX tests with only two possible results use 1/2 instead of 1/6. 


Oct 30 2003, 06:21
Post
#4


ABC/HR developer, ff123.net admin Group: Developer (Donating) Posts: 1396 Joined: 24September 01 Member No.: 12 
The simplest way to do it is to simulate it:
http://ff123.net/abx/abx.html You don't have to remember, derive, or look up all those messy equations. Plus, that's the way things happen in real life anyway. ff123 


Oct 30 2003, 07:28
Post
#5


Group: Members Posts: 391 Joined: 24December 02 From: Eugene, OR Member No.: 4224 
QUOTE (sshd @ Oct 29 2003, 08:32 PM) QUOTE (phong @ Oct 30 2003, 04:23 AM) whatever I admint I am not an expert, but the socalled Pvalue probably refers to the Greek letter Rho, which looks pretty much like a P. Rho stands for probability. It is quite easy to calculate the value. Say you have a normal 6 edged dice. The probability (Rho) of getting a 6 in the first try is 1/6 ~ 16,7%. The probability (Rho) of getting 6 in the fist try and 2 in the second try is 1/6 * 1/6 ~ 2,7%. When you are doing simple ABX tests with only two possible results use 1/2 instead of 1/6. I'm pretty sure that's not it. The Pvalue is finding the odds that something occured by chance alone, without any other external factors.  http://www.pkulak.com



Oct 30 2003, 08:22
Post
#6


Group: Members Posts: 473 Joined: 7June 02 Member No.: 2244 
QUOTE (torok @ Oct 30 2003, 08:28 AM) I'm pretty sure that's not it. The Pvalue is finding the odds that something occured by chance alone, without any other external factors. The pvalue is the probability that the test is passed under the condition that you were only guessing. It is not the probability that you were guessing. (Not sure if you meant this anyway.) And if you don't speak Python (like me): CODE n! := 1*2*3*...*n (n>0)
0! := 1 a! binomial(a,b) :=  (a>=b) b! (ab)! trials  binomial(trials,k) pval := \  / trials  2 k=correct This post has been edited by Continuum: Oct 30 2003, 08:23 


Oct 30 2003, 13:24
Post
#7


Moderator Group: Super Moderator Posts: 3936 Joined: 29September 01 Member No.: 73 
QUOTE (Continuum @ Oct 30 2003, 08:22 AM) The pvalue is the probability that the test is passed under the condition that you were only guessing. Why does the old PCABX program gives a 100% probability for one success out of one test ? It should be 50 %. 


Oct 30 2003, 14:50
Post
#8


Group: Members Posts: 473 Joined: 7June 02 Member No.: 2244 
QUOTE (Pio2001 @ Oct 30 2003, 02:24 PM) QUOTE (Continuum @ Oct 30 2003, 08:22 AM) The pvalue is the probability that the test is passed under the condition that you were only guessing. Why does the old PCABX program gives a 100% probability for one success out of one test ? It should be 50 %. No idea, I never used it. 50% is correct. I assume the calculations of PCABX are correct for more trials? 


Oct 30 2003, 15:02
Post
#9


Group: Members Posts: 97 Joined: 19September 03 From: BKK, Thailand Member No.: 8916 
Continuum's definition of PValue for ABX test is correct. (binomial test; h0: you are guessing [p=0.5], h1: you are not guessing )
And I think PCABX just use wrong formular. 


Oct 30 2003, 18:35
Post
#10


Java ABC/HR developer Group: Developer Posts: 175 Joined: 17September 03 Member No.: 8879 
Just for the sake of completeness, here's the method I use in abchr/java:
CODE public static double pVal(int n,int correct) { double result=0; double binom; int twos; for(int k=correct;k<=n;k++) { twos=n; binom=1; for(int j=0;j<k;j++) { binom*=(nj)/(double)(kj); while(binom>2 && twos>0){binom/=2;twos;} } for(int j=0;j<twos;j++){binom/=2;} result+=binom; } return result; } Conceptwise, it's exactly the same as the method phong uses, although not as straightforward. The advantage is mainly that it's probably a bit faster and it doesn't overflow during your casual 170+ trials ABX test I don't know exactly when it does overflow, but I tested it once for 10000 trials with 5000 correct and it got the right result after about 5 minutes of calculating (well, this is a scientific forum, you know.. every last 0.0001% of certainty in an ABX test is valued ) When it comes to optimizing programs I'm a total amateur, though (I rarely do much optimizing in my programs), so there are probably much better ways to calculate that. This post has been edited by schnofler: Oct 30 2003, 18:36 


Oct 30 2003, 19:10
Post
#11


ABC/HR developer, ff123.net admin Group: Developer (Donating) Posts: 1396 Joined: 24September 01 Member No.: 12 
In my version of abchr, I don't bother calculating something if it's not going to change the final answer by a certain amount (p values < 0.001 are not reported to their full precision). This skirts both the underflow and speed issues.
ff123 


Oct 30 2003, 19:24
Post
#12


Java ABC/HR developer Group: Developer Posts: 175 Joined: 17September 03 Member No.: 8879 
QUOTE In my version of abchr, I don't bother calculating something if it's not going to change the final answer by a certain amount (p values < 0.001 are not reported to their full precision). This skirts both the underflow and speed issues. Well, that might be true. But as a math student, I take pride in considering this unprofessional. 


Oct 30 2003, 21:26
Post
#13


ABC/HR developer, ff123.net admin Group: Developer (Donating) Posts: 1396 Joined: 24September 01 Member No.: 12 
QUOTE (schnofler @ Oct 30 2003, 10:24 AM) QUOTE In my version of abchr, I don't bother calculating something if it's not going to change the final answer by a certain amount (p values < 0.001 are not reported to their full precision). This skirts both the underflow and speed issues. Well, that might be true. But as a math student, I take pride in considering this unprofessional. Haha! Since I'm an engineer, I take that as a compliment. ff123 


Oct 30 2003, 22:18
Post
#14


Java ABC/HR developer Group: Developer Posts: 175 Joined: 17September 03 Member No.: 8879 
QUOTE (ff123 @ Oct 30 2003, 12:26 PM) QUOTE (schnofler @ Oct 30 2003, 10:24 AM) QUOTE In my version of abchr, I don't bother calculating something if it's not going to change the final answer by a certain amount (p values < 0.001 are not reported to their full precision). This skirts both the underflow and speed issues. Well, that might be true. But as a math student, I take pride in considering this unprofessional. Haha! Since I'm an engineer, I take that as a compliment. ff123 LOL! Who said there can be no holy wars in a perfectly scientific discussion? 


LoFi Version  Time is now: 24th October 2014  18:15 