IPB

Welcome Guest ( Log In | Register )

Minimum number of required ABX trials, Split from from topic ID: 92851
sauvage78
post Jan 12 2012, 00:41
Post #1





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



The minimal number of trials depends on how successfull you are.
How quickly you are sucessfull shows how confident in yourself you are.

The time & number of sucessfull trials are tied, you should never separate them when judging an ABX log.

With F2K ABX component, 8 sucessfull trials in a row (& if all successfull in a row, it usually means quick) trials is the minimum for me.

As soon as you begin to fail you can easyly increase to 10 or 12 to try to "erease" your failures.
In this case if you fail once or twice you can usually still get a signifiant result although it usually means the ABXing was hard, & by consequence longer as you begin to hesitate.

Usually if you begin to fail more than 3 times on 12 trials, it begins to be so hard & you have so much hesitation that it begins to take forever to ABX. At this stage I usually give up by myself & declare that I cannot ABX as in general it means I am not sure that the audio part I am focusing on actually contains any real artefact.

This post has been edited by sauvage78: Jan 12 2012, 00:42


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
 
Start new topic
Replies
sauvage78
post Jan 12 2012, 04:54
Post #2





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



QUOTE
Are you claiming that I (or greynol) have not performed ABX tests?

I don't know, as I said I don't trust anyone until I have double tested him. I just didn't double test neither saratoga or greynol. Maybe there are logs floating around. I don't know & don't care.

But I wouldn't worry about that if I were you because it happens that if I recall well I have already disagreed about vorbis vs nero quality (I think Nero AAC was better) with IgorC in the past. So for me the "credibility" of IgorC is not "perfect" as we don't always agree together. Does this mean he is wrong or dishonest, does this mean I am better than him ? No, it means ABXing is not a perfect science.

So in the end my only real objection to you & Greynol is & has always been that "perfectly" ABXing takes time. That perfection doesn't exist when ABXing & that knowing this fact anyone who seriously wants to ABX have to make compromise between theory & practice & take the middle road.

It's one thing to post 2 or 3 logs with 16 trials from time to time. It's another thing when you want to create a big test & when you try to run a big test you understand that you have to lower your requirement & make compromise in order for the test to be physically do-able.

In the end all I am saying is the fact that telling a newbie that 16 trials is the solution to his 5 trials problem is not the solution. If 5 trials is too low to convince others, you do not have to publish test with 16 trials if you think it is a waste of time. There a sweat spot in between 5 & 16 that should be enough to convince others of your honnesty, & if others are not satisfied with 8 to 12 trials depending on your success (or whatever is your personnal methodology) ... as long as you conform to the TOS ... well let them go to hell, you did your best.


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
saratoga
post Jan 12 2012, 05:02
Post #3





Group: Members
Posts: 5152
Joined: 2-September 02
Member No.: 3264



QUOTE (sauvage78 @ Jan 11 2012, 22:54) *
QUOTE
Are you claiming that I (or greynol) have not performed ABX tests?

I don't know, as I said I don't trust anyone until I have double tested him. I just didn't double test neither saratoga or greynol. Maybe there are logs floating around. I don't know & don't care.


If you don't know, its probably better not to make up things about people. Some might consider that dishonest.

QUOTE (sauvage78 @ Jan 11 2012, 22:54) *
So in the end my only real objection to you & Greynol is & has always been that "perfectly" ABXing takes time. That perfection doesn't exist when ABXing & that knowing this fact anyone who seriously wants to ABX have to make compromise between theory & practice & take the middle road.


"Doing this correctly is hard" is not an objection, its a cop-out. If audio were an easy thing, people wouldn't make a hobby or a science out of it. But its not easy, and so we do. If you dislike how difficult it is, take up fishing smile.gif

QUOTE (sauvage78 @ Jan 11 2012, 22:54) *
In the end all I am saying is the fact that telling a newbie that 16 trials is the solution to his 5 trials problem is not the solution. If 5 trials is too low to convince others, you do not have to publish test with 16 trials if you think it is a waste of time. There a sweat spot in between 5 & 16 that should be enough to convince others of your honnesty, & if others are not satisfied with 8 to 12 trials depending on your success (or whatever is your personnal methodology) ... as long as you conform to the TOS ... well let them go to hell, you did your best.


Funny, I said:

QUOTE (saratoga)
Looking at this test, I think the OP did more trials then he needed to, but he did them of the wrong samples. It seems like he only thinks he can ABX one or two of the 10 samples, so the obvious thing to do would have been to do about 10-12 trials of each of those few samples.


So no disagreement from me!
Go to the top of the page
+Quote Post

Posts in this topic
- sauvage78   Minimum number of required ABX trials   Jan 12 2012, 00:41
- - greynol   It is expected that you choose the number of trial...   Jan 12 2012, 00:44
- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 18:41) Th...   Jan 12 2012, 00:45
- - sauvage78   Well there is the theory & there is real life ...   Jan 12 2012, 00:56
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 18:56) To...   Jan 12 2012, 01:05
- - greynol   He should be conducting sets of 16 trials at first...   Jan 12 2012, 01:10
- - sauvage78   I never said I judged this test valid, I only gave...   Jan 12 2012, 01:13
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 19:13) I ...   Jan 12 2012, 01:17
|- - greynol   QUOTE (sauvage78 @ Jan 11 2012, 16:13) I ...   Jan 12 2012, 01:20
- - sauvage78   saratoga: Yes, I had the feeling that you were thi...   Jan 12 2012, 01:27
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 19:27) Ye...   Jan 12 2012, 01:32
|- - greynol   QUOTE (saratoga @ Jan 11 2012, 16:32) del...   Jan 12 2012, 01:42
- - sauvage78   Well I know this topic isn't about me but my...   Jan 12 2012, 02:31
- - greynol   The difference between you and the OP is that you ...   Jan 12 2012, 02:54
- - sauvage78   I don't even need a log anymore to trust /mnt ...   Jan 12 2012, 03:30
- - IgorC   A lot of discussion here but it won't change t...   Jan 12 2012, 03:55
|- - saratoga   QUOTE (IgorC @ Jan 11 2012, 21:55) A lot ...   Jan 12 2012, 04:06
- - sauvage78   Even if I think 5 trials is too low to convince ot...   Jan 12 2012, 04:01
|- - greynol   QUOTE (sauvage78 @ Jan 11 2012, 19:01) Ig...   Jan 12 2012, 04:13
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 22:01) Ev...   Jan 12 2012, 04:13
- - IgorC   I think I understand what sauvage78 wants to say. ...   Jan 12 2012, 04:46
|- - saratoga   QUOTE (IgorC @ Jan 11 2012, 22:46) It...   Jan 12 2012, 04:50
|- - IgorC   QUOTE (saratoga @ Jan 12 2012, 00:50) QUO...   Jan 12 2012, 04:57
- - sauvage78   QUOTE Are you claiming that I (or greynol) have no...   Jan 12 2012, 04:54
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 22:54) QU...   Jan 12 2012, 05:02
|- - IgorC   QUOTE (sauvage78 @ Jan 12 2012, 00:54) It...   Jan 12 2012, 05:03
|- - greynol   QUOTE (IgorC @ Jan 11 2012, 20:03) So it...   Jan 12 2012, 05:19
- - sauvage78   IgorC: I was more trying to say that if TOS8 is ve...   Jan 12 2012, 05:02
|- - IgorC   QUOTE (sauvage78 @ Jan 12 2012, 01:02) Ig...   Jan 12 2012, 05:06
- - nesf   From a complete newbie perspective: It could have ...   Jan 12 2012, 11:08
|- - saratoga   QUOTE (nesf @ Jan 12 2012, 05:08) From a ...   Jan 12 2012, 23:15
- - apodtele   Please read Fallacy of p-value. I just want to po...   Jan 12 2012, 17:09
|- - Porcus   QUOTE (apodtele @ Jan 12 2012, 17:09) Ple...   Jan 12 2012, 22:07
- - krabapple   It looks like we're groping towards a discussi...   Jan 12 2012, 18:00
- - nesf   A Dummies guide for doing some basic two sample an...   Jan 13 2012, 01:59


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st December 2014 - 02:23