Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: What about a more selection-/cheatproof ABXer? (Read 1646 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

What about a more selection-/cheatproof ABXer?

For public listening tests, what about:

- publickey.txt that fb2k can read (hey, could even be installed as a component)
- write ABX results to file, encrypted with the public key

?

Reason: Bringing the procedure closer to double-blind.
As of today, ABXing with voluntary submission of results, is like first blinding patient and doctor, and then showing the result to the patient with the question of whether this should be submitted. (Would a patient who has had positive effects of something that turns out to be placebo, be equally likely to submit?)

For listening tests, it might be that people are less likely to submit results with negative findings. Especially if they score less than 50%. Indeed, people might just think the latter is nonsense and discard. Could very well be due to ignorance and neglect, not to deliberate cheating.

Suggestion might guard against those selection biases. (Although, not against deliberate manipulation, where you  (i) ABX  the normal way, and then for the more-doubleblind-test (ii) choose to test only those where you previously did well.)

What about a more selection-/cheatproof ABXer?

Reply #1
The owner of the private key would need to be trusted with all the results, and we should require them to publish them all after they've stopped taking submissions (or, automatically upon submission, if there's no date limit).