IPB

Welcome Guest ( Log In | Register )

14 Pages V   1 2 3 > »   
Reply to this topicStart new topic
What's the problem with double-blind testing?
mirrorsawlljk
post Oct 19 2005, 03:03
Post #1





Group: Members
Posts: 11
Joined: 15-October 05
Member No.: 25138



I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable. I'm afraid I don't understand his angle of attack. I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test.

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing. Is there an audio magazine that isn't filled with this kind of thinking?
Go to the top of the page
+Quote Post
bubka
post Oct 19 2005, 03:07
Post #2





Group: Members
Posts: 239
Joined: 21-July 02
Member No.: 2692



some people can actually detect specific codecs by their sounds, which defeats the purpose, but there is no problem, its as scientific as you can get.

Does this person also believe the earth was created in 6 days too?


--------------------
Chaintech AV-710
Go to the top of the page
+Quote Post
TheQat
post Oct 19 2005, 03:27
Post #3





Group: Members
Posts: 41
Joined: 20-October 02
Member No.: 3579



QUOTE (mirrorsawlljk @ Oct 18 2005, 06:03 PM)
I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable.  I'm afraid I don't understand his angle of attack.  I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test. 

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing.  Is there an audio magazine that isn't filled with this kind of thinking?
*


Keep in mind that if Stereophile's audience realizes that double-blinds are totally valid and acknowledges the importance of that realization, all of Stereophile's ad revenue will evaporate as companies that make massively over-priced cable (I bet you can find several full-page ads for cable in whatever issue you happen to be reading) and other hocus-pocus products go out of business like medieval humans dying of the black plague.

Therefore it is in the best economic interest of Stereophile and all its writers to encourage the entirely wrong-headed thinking that double-blind testing is unreliable, and as apparently unscrupulous people they do so and will continue to do so.

Edit: Hello goon smile.gif
Edit2: I too would like to know if there is an audio magazine that is not in love with products of dubious worth.

This post has been edited by TheQat: Oct 19 2005, 03:32
Go to the top of the page
+Quote Post
Tahnru
post Oct 19 2005, 04:29
Post #4





Group: Members
Posts: 325
Joined: 17-October 05
From: United States
Member No.: 25178



The closest thing I have seen to a legitimate criticism fielded when discussing double blind tests was a discussion about how far the results were interpreted. Of course, this didn't address the validity of the testing method at all, but only the misinterpretation or overinterpretation of the results. And even that doesn't seem to be a problem very often.
Go to the top of the page
+Quote Post
singaiya
post Oct 19 2005, 04:31
Post #5





Group: Members
Posts: 365
Joined: 21-November 02
Member No.: 3830



QUOTE (TheQat @ Oct 18 2005, 06:27 PM)
QUOTE (mirrorsawlljk @ Oct 18 2005, 06:03 PM)
I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable.  I'm afraid I don't understand his angle of attack.  I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test. 

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing.  Is there an audio magazine that isn't filled with this kind of thinking?
*


Keep in mind that if Stereophile's audience realizes that double-blinds are totally valid and acknowledges the importance of that realization, all of Stereophile's ad revenue will evaporate as companies that make massively over-priced cable (I bet you can find several full-page ads for cable in whatever issue you happen to be reading) and other hocus-pocus products go out of business like medieval humans dying of the black plague.

Therefore it is in the best economic interest of Stereophile and all its writers to encourage the entirely wrong-headed thinking that double-blind testing is unreliable, and as apparently unscrupulous people they do so and will continue to do so.

Edit: Hello goon smile.gif
Edit2: I too would like to know if there is an audio magazine that is not in love with products of dubious worth.
*




That's the answer. Stereophile as a magazine is not unique in its reliance on advertising revenue. Since all magazines rely on advertising, and advertisers in this scenario rely on selling things like cables that cannot be double blind tested positively, there is unlikely to be any "high end" audio mags willing to promote double blind testing as a valid procedure for evaluation. The mags and the advertisers have a sort of symbiotic relationship; they need each other. DBTs are bad for business all round.

Maybe there are some more modern computer/digital audio oriented magazines out there that have a different segment of advertisers who are not threatened by DBT, but I don't know of any.
Go to the top of the page
+Quote Post
Axon
post Oct 19 2005, 07:06
Post #6





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



The Audio Critic is notably pro-DBT.
Go to the top of the page
+Quote Post
onthejazz
post Oct 19 2005, 10:03
Post #7





Group: Members
Posts: 117
Joined: 2-April 02
From: Minnesota
Member No.: 1666



Interesting publication. I like it, too bad its not in full production anymore. Thanks much for the link. Anybody have any more quality sources?
Go to the top of the page
+Quote Post
Donunus
post Oct 19 2005, 11:00
Post #8





Group: Members
Posts: 226
Joined: 8-July 05
Member No.: 23210



The writer of the letter in stereophile is incorrect in saying double blind tests are unreliable but he has a point. I do abx tests in foobar to compare different files a lot but there just are too many songs to abx to make sure all of one formats files are as good as the other ex. mp3 vs aac vs wav, etc... The thing I always notice though is that when using the analytical side of the brain(abx testing) only certain parts of the music is being focused and the emotional side of the music isn't being analyzed, therefore the goosebump factor of the sound the way the music delivers emotion cannot be analyzed! Its very hard to have the left and right side of the brain working at the same time.

My Example for this is when doing an abx of one particular song, I passed the test in foobar with flying colors(100% on abx test) but thought the differences were not really significant enough to make me keep the wav files from the cds and kept only the mp3s. After a few weeks of listening to the mp3s and really getting to know and love the songs better, I tried just popping the cd in for a listen... wow, goosebumps.... The parts I loved in the songs i was listening to gave me goosebumps for the first time. Now thats music! The feeling is lost on some parts of the mp3. And that was with my pc using sennheisers, not even my high end home rig.

Its not totally reliable to trust abx testing for determining the enjoyment one gets from his music because human memory can only remember only a certain number of seconds at a time like in an abx test. Its good for making codecs for eliminating artifacts but abx testing a few seconds of one song does not determine the accuracy of the dynamics of the whole piece. All the buildup of sound, the emotion! Bash me now but I have been an audiophile for more than 15 years and I am only 31, and have been a music lover since 5 years old. I know what I'm talking about. Listening to a bad stereo system, I can hear differences between different files for example but I don't really tend to care whether I am listening to the mp3 or the wav version. But with a very involving music system, the mp3 can sometimes put a damper on the fun factor of the song.

Also, stereophile loves posting "gray area" letters. I love that magazine. They exaggerate things sometimes but overall they are very accurate the way they subjectively describe the audio quality of equipment.
Go to the top of the page
+Quote Post
KikeG
post Oct 19 2005, 11:54
Post #9


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



The "goosebump", emotional factor can be caused by placebo effect, you can't be sure that is not the cause. People has used same principle (emotional response and the like) to say cables sound different, for example.

You can do long-term blind tests too, not just the quick-switch, short snippet, typical abx test.

This post has been edited by KikeG: Oct 19 2005, 11:55
Go to the top of the page
+Quote Post
Donunus
post Oct 19 2005, 12:06
Post #10





Group: Members
Posts: 226
Joined: 8-July 05
Member No.: 23210



QUOTE (KikeG @ Oct 19 2005, 06:54 PM)
The "goosebump", emotional factor can be caused by placebo effect, you can't be sure that is not the cause. People has used same principle (emotional response and the like) to say cables sound different, for example.

You can do long-term blind tests too, not just the quick-switch, short snippet, typical abx test.
*

What do you mean by long term abx? how do I do this? Play entire songs? actually can you give me a link that explains placebo. thx
Go to the top of the page
+Quote Post
PoisonDan
post Oct 19 2005, 12:18
Post #11





Group: Members (Donating)
Posts: 678
Joined: 10-December 01
From: Belgium
Member No.: 622



QUOTE (Axon @ Oct 19 2005, 08:06 AM)
The Audio Critic is notably pro-DBT.
*

Thanks. I really like their sample article "The Ten Biggest Lies in Audio".


--------------------
Over thinking, over analyzing separates the body from the mind.
Go to the top of the page
+Quote Post
kjoonlee
post Oct 19 2005, 18:31
Post #12





Group: Members
Posts: 2526
Joined: 25-July 02
From: South Korea
Member No.: 2782



QUOTE (Donunus @ Oct 19 2005, 08:06 PM)
actually can you give me a link that explains placebo. thx
*

The word placebo, when used in audio circles, is a little different from the medical term, where people can actually benefit a little from dummy treatments.

According to ff123, it's closer to expectation bias, where a person's expectation can influence his perception, if I understand correctly.


--------------------
http://blacksun.ivyro.net/vorbis/vorbisfaq.htm
Go to the top of the page
+Quote Post
KikeG
post Oct 19 2005, 19:33
Post #13


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



Well, I think expectation effects in listening tests actually make people perceive the sound as truly different, so they cause an equivalent effect to the placebo effect in medicine. I mean, both them cause a real effect: patients think that they are having a medicine, and that cures them. Listeners think that their equipment improves or deteriorates the sound, and that makes them truly perceive these effects.

This post has been edited by KikeG: Oct 19 2005, 19:52
Go to the top of the page
+Quote Post
krabapple
post Oct 19 2005, 20:06
Post #14





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



QUOTE (mirrorsawlljk @ Oct 18 2005, 06:03 PM)
I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable.  I'm afraid I don't understand his angle of attack.  I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test. 

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing.  Is there an audio magazine that isn't filled with this kind of thinking?
*


By all means read it..it's always good for a laugh. I have the current issue at home and am waiting for a free hour or two to enjoy it.

I've interacted with editor John Atkinson online and very briefly in person -- he's mastered the art of balancing himself on a rehtorical tightrope between scientific evaluation of claims (he has a background in physics) and audiofoolery. This manifests itself in his magazine as the schizophrenic devotion to engineering jargon and inclusion of bench measurements -- sometimes extremely detailed -- accompanying sighted 'reviews' that are literally whimsical. Atkinson knows who his subscriber base is and I doubt he's about to alienate them by embracing valid scientific methods for subjective comparison...ever. The purely 'subjective' experience will ALWAYS trump ANY measurement or scientific method, to that crowd.

But Stereophile is a model of restraint compared to The Absolute Sound, which is absolutely off the deep end, and has been forever.

Peter Aczel's 'The Audio Critic' (now online only) is the closest thing to a purely objective audio magazine you'll find. (www.theaudiocritic.com). Lesser lights include The Sensible Sound (about half of the people there seem to drink the hi-end cool-aid) and Sound & Vision (David Ranada is a DBT advocate, Ian MAsters is a no-nonsense objectivist, Ken Pohlmann is a god of digital, but the mag as a whole seems ever more tilted towards marketing hype -- I don't see much by Ranada in there recently)

This post has been edited by krabapple: Oct 19 2005, 20:15
Go to the top of the page
+Quote Post
krabapple
post Oct 19 2005, 20:12
Post #15





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



QUOTE (KikeG @ Oct 19 2005, 10:33 AM)
Well, I think expectation effects in listening tests actually make people perceive the sound as truly different, so they cause an equivalent effect to the placebo effect in medicine. I mean, both them cause a real effect: patients think that they are having a medicine, and that cures them. Listeners think that their equipment improves or deteriorates the sound, and that makes them truly perceive these effects.
*


The audio placebo effect also makes people 'feel' differently, and that doubtless has some neurophysiological correlate -some measureable brain activity correlated to the feeling -- so it's 'real' in that sense. But the point is it's entirely self-generated -- it is not due to some real difference between sounds. (Just as the beneficial medical placebo effect isn't 'due', in any meaningful way, to the sugar pill itself -- it's due to the idea that the sugar pill was medicine.)

This post has been edited by krabapple: Oct 19 2005, 20:16
Go to the top of the page
+Quote Post
Danimal
post Oct 19 2005, 20:20
Post #16





Group: Members
Posts: 132
Joined: 16-June 03
From: Ohio
Member No.: 7209



QUOTE (Donunus @ Oct 19 2005, 05:00 AM)
The writer of the letter in stereophile is incorrect in saying double blind tests are unreliable but he has a point. I do abx tests in foobar to compare different files a lot but there just are too many songs to abx to make sure all of one formats files are as good as the other ex. mp3 vs aac vs wav, etc... The thing I always notice though is that when using the analytical side of the brain(abx testing) only certain parts of the music is being focused and the emotional side of the music isn't being analyzed, therefore the goosebump factor of the sound the way the music delivers emotion cannot be analyzed! Its very hard to have the left and right side of the brain working at the same time.

My Example for this is when doing an abx of one particular song, I passed the test in foobar with flying colors(100% on abx test) but thought the differences were not really significant enough to make me keep the wav files from the cds and kept only the mp3s. After a few weeks of listening to the mp3s and really getting to know and love the songs better, I tried just popping the cd in for a listen... wow, goosebumps.... The parts I loved in the songs i was listening to gave me goosebumps for the first time. Now thats music! The feeling is lost on some parts of the mp3. And that was with my pc using sennheisers, not even my high end home rig.

Its not totally reliable to trust abx testing for determining the enjoyment one gets from his music because human memory can only remember only a certain number of seconds at a time like in an abx test. Its good for making codecs for eliminating artifacts but abx testing a few seconds of one song does not determine the accuracy of the dynamics of the whole piece. All the buildup of sound, the emotion! Bash me now but I have been an audiophile for more than 15 years and I am only 31, and have been a music lover since 5 years old. I know what I'm talking about. Listening to a bad stereo system, I can hear differences between different files for example but I don't really tend to care whether I am listening to the mp3 or the wav version. But with a very involving music system, the mp3 can sometimes put a damper on the fun factor of the song.

Also, stereophile loves posting "gray area" letters. I love that magazine. They exaggerate things sometimes but overall they are very accurate the way they subjectively describe the audio quality of equipment.
*


The point of double-blind testing is to make sure that you're getting goosebumps over something you actually hear, and not something you expect to hear. The whole point is to try to evaluate what you're hearing without knowing whether it's the original or the lossy encode. That shouldn't remove the "emotional" part of it, just make sure that you're reacting based upon something that is really there, and not what you want to hear or what you expect to hear.
Go to the top of the page
+Quote Post
Halcyon
post Oct 19 2005, 20:29
Post #17





Group: Members
Posts: 244
Joined: 6-November 01
Member No.: 416



There is nothing inherently bad about double blind.

However, 'double blind' is not a test method per se, it's a test methodology requirement (or super class) to rule out bias.

ABCHR is a rough outline of a test methodology in sensory evaluation.

So is ABX.

Hoever, even they can be set up (i.e. implemented) improperly and used to produce useless tests (that prove nothing).

For example, if you set up a test for ABX with persons who are CONVINCED they will hear NO difference, you have a strong tester bias that a simple ABX test method (even when double blind) does not protect against.

There are many inherently problematic issues about sensory evaluation of stimuli at the treshold of detection.

Currently, most set ups crudely ignore this and often err on the side of ruling out type II errors and as such, increasing the likelihood of type I errors in the test setup. This is esp. true of most statistical "analysis" (it's not really analysis, it's crude mechanistic calculation) of the test data, when you look at most of the published studies.

The problem is that many people fail to make a conceptual distinction between the following concepts:

- Double blind test
- ABX test
- Sensory evaluation test

They are not all interchangeable and correct procedure in one, does not necessarily translate to a correct procedure in another.

This post has been edited by Halcyon: Oct 19 2005, 20:30
Go to the top of the page
+Quote Post
ff123
post Oct 19 2005, 20:53
Post #18


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



Tests that try to distinguish very small effects *and* aim for small type I and type II errors either require lots of listeners or lots of trials. See my test sensitivity calculator:

http://ff123.net/export/TestSensitivityAnalyzer.xls

Column "D" shows the Proportion of Distinguishers, that is, the proportion of listeners in a randomly sampled group of people who would be expected to hear a difference in an audio comparison. The smaller the proportion of distinguishers, the smaller the effect. If the proportion of distinguishers is 30% (a pretty small effect), you'd need 119 listeners out of which 69 have to make a correct response to limit both the type I and type II errors to 0.05. I assume this would be the criteria that would satisfy both the "objective" and the "subjective" crowds.

But obviously there is already a huge problem in trying to assemble that many listeners or in attempting such a large number of trials.

ff123
Go to the top of the page
+Quote Post
fcmts
post Oct 19 2005, 22:15
Post #19





Group: Members
Posts: 9
Joined: 21-March 04
Member No.: 12882



There is another problem similar to wine blind tests: it is very hard to believe that wine experts can't recognize the origin of the samples by the taste. Although those tests are not abx, that bias is also present in sound abx tests for the testers are, in general, experts in codecs.
Go to the top of the page
+Quote Post
stephanV
post Oct 19 2005, 22:22
Post #20





Group: Members
Posts: 394
Joined: 6-May 04
Member No.: 13932



I don't see how that is a problem. The goal of an ABX test is to determine if there is an audible difference, it is not intended to determine the quality of a track. The answer is "yes" or "no", not "good" or "bad" as in wine tests.


--------------------
"We cannot win against obsession. They care, we don't. They win."
Go to the top of the page
+Quote Post
Yaztromo
post Oct 19 2005, 23:05
Post #21





Group: Members
Posts: 236
Joined: 28-July 03
From: England, UK
Member No.: 8031



QUOTE (TheQat @ Oct 19 2005, 03:27 AM)
Edit: Hello goon smile.gif

*


How in the world of intuition did you figure that bit out? I don't see no stairs!

This post has been edited by Yaztromo: Oct 19 2005, 23:06
Go to the top of the page
+Quote Post
krabapple
post Oct 19 2005, 23:40
Post #22





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



QUOTE (ff123 @ Oct 19 2005, 11:53 AM)
Tests that try to distinguish very small effects *and* aim for small type I and type II errors either require lots of listeners or lots of trials.  See my test sensitivity calculator:

http://ff123.net/export/TestSensitivityAnalyzer.xls

Column "D" shows the Proportion of Distinguishers, that is, the proportion of listeners in a randomly sampled group of people who would be expected to hear a difference in an audio comparison.  The smaller the proportion of distinguishers, the smaller the effect.  If the proportion of distinguishers is 30% (a pretty small effect), you'd need 119 listeners out of which 69 have to make a correct response to limit both the type I and type II errors to 0.05.  I assume this would be the criteria that would satisfy both the "objective" and the "subjective" crowds.

But obviously there is already a huge problem in trying to assemble that many listeners or in attempting such a large number of trials.

ff123
*



Interesting. Check out this AES preprint about a comparison SACD and DVD-A , whihc seems to be derived from a German student's masters thesis:

http://www.hfm-detmold.de/eti/projekte/dip..._paper_6086.pdf

Four out of 45 listeners comparing stereo recordings passed an ABX test. None of 100 listeners (include the four mentioned above) passed an ABX using surround material. The material listened to by each of the four 'successful' listeners was different. Also, they all onl;y heard the difference using headphones. There may have been some switch noise affecting the results.

Can you make any evaluation of the senstivity of these tests?

This post has been edited by krabapple: Oct 19 2005, 23:40
Go to the top of the page
+Quote Post
Donunus
post Oct 20 2005, 01:51
Post #23





Group: Members
Posts: 226
Joined: 8-July 05
Member No.: 23210



QUOTE (Danimal @ Oct 20 2005, 03:20 AM)
QUOTE (Donunus @ Oct 19 2005, 05:00 AM)
The writer of the letter in stereophile is incorrect in saying double blind tests are unreliable but he has a point. I do abx tests in foobar to compare different files a lot but there just are too many songs to abx to make sure all of one formats files are as good as the other ex. mp3 vs aac vs wav, etc... The thing I always notice though is that when using the analytical side of the brain(abx testing) only certain parts of the music is being focused and the emotional side of the music isn't being analyzed, therefore the goosebump factor of the sound the way the music delivers emotion cannot be analyzed! Its very hard to have the left and right side of the brain working at the same time.

My Example for this is when doing an abx of one particular song, I passed the test in foobar with flying colors(100% on abx test) but thought the differences were not really significant enough to make me keep the wav files from the cds and kept only the mp3s. After a few weeks of listening to the mp3s and really getting to know and love the songs better, I tried just popping the cd in for a listen... wow, goosebumps.... The parts I loved in the songs i was listening to gave me goosebumps for the first time. Now thats music! The feeling is lost on some parts of the mp3. And that was with my pc using sennheisers, not even my high end home rig.

Its not totally reliable to trust abx testing for determining the enjoyment one gets from his music because human memory can only remember only a certain number of seconds at a time like in an abx test. Its good for making codecs for eliminating artifacts but abx testing a few seconds of one song does not determine the accuracy of the dynamics of the whole piece. All the buildup of sound, the emotion! Bash me now but I have been an audiophile for more than 15 years and I am only 31, and have been a music lover since 5 years old. I know what I'm talking about. Listening to a bad stereo system, I can hear differences between different files for example but I don't really tend to care whether I am listening to the mp3 or the wav version. But with a very involving music system, the mp3 can sometimes put a damper on the fun factor of the song.

Also, stereophile loves posting "gray area" letters. I love that magazine. They exaggerate things sometimes but overall they are very accurate the way they subjectively describe the audio quality of equipment.
*


The point of double-blind testing is to make sure that you're getting goosebumps over something you actually hear, and not something you expect to hear. The whole point is to try to evaluate what you're hearing without knowing whether it's the original or the lossy encode. That shouldn't remove the "emotional" part of it, just make sure that you're reacting based upon something that is really there, and not what you want to hear or what you expect to hear.
*



Well, with mp3, even 320. I have passed foobar abx with many samples from my own cds compared to the original with flying colors. What I am saying is I cannot get goosebumps while I am in the analyzing mode cause I am not really listening to the music but to different aspects of the sound while doing the abx test. I do hear the differences and although they are not really that big from an analysis standpoint, they become bigger when listening for enjoyment. Some songs will just lose some life when encoded to mp3.
Go to the top of the page
+Quote Post
ff123
post Oct 20 2005, 05:38
Post #24


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (krabapple @ Oct 19 2005, 02:40 PM)
Interesting.  Check out this AES preprint about a comparison SACD and DVD-A , whihc seems to be derived from a German student's masters thesis:

http://www.hfm-detmold.de/eti/projekte/dip..._paper_6086.pdf
Four out of 45 listeners comparing stereo recordings passed an ABX test.  None of 100 listeners (include the four mentioned above) passed an ABX using surround material.  The material listened to by each of the four 'successful' listeners was different.  Also, they all onl;y heard the difference using headphones.  There may have been some switch noise affecting the results.

Can you make any evaluation of the senstivity of these tests?
*


Proponent of one format vs the other: "Aha, 4 people definitely heard a difference!" and "That low-level crackling sound when switching didn't affect the test's validity."

Someone in the no-difference camp: "Aha, 141 out of 145 20-run ABX tests did not show a difference!"

Choose your interpretation.

ff123
Go to the top of the page
+Quote Post
Lyx
post Oct 20 2005, 06:07
Post #25





Group: Members
Posts: 3353
Joined: 6-July 03
From: Sachsen (DE)
Member No.: 7609



QUOTE (Donunus @ Oct 19 2005, 01:06 PM)
QUOTE (KikeG @ Oct 19 2005, 06:54 PM)
The "goosebump", emotional factor can be caused by placebo effect, you can't be sure that is not the cause. People has used same principle (emotional response and the like) to say cables sound different, for example.

You can do long-term blind tests too, not just the quick-switch, short snippet, typical abx test.
*

What do you mean by long term abx? how do I do this? Play entire songs?
*


Well, this is a two-sided sword, because long-term DBTs are indeed theoretically no problem - but in practice, most ABX-tools aren't designed for this purpose, which makes a bit more difficult to setup.

It basically goes like this - any abx-test may take as long as you want - there is no limit on how long the timespan has to be between trials - there is no limit in how long the entire test takes - and there is no limit on the number of trials.

What this means is, that you could for example do a 6months long DBT, where you just pick one of your fav-albums, and just go your normal daily schedule.... BUT the application which you use for playback should make it invisible to you if the original or the encoded version is currently played. Then, during those 6 months, when you just normally listen to that fav-album, and think "this is the encoded version!", then you just click a button, and afterwards continue your normal daily life. Over the course of the whole 6 months, you repeat that a dozen times.... and afterwards you end up with a longterm DBT which has so many trials, that its accuracy is incredibly high.

The only thing to keep in mind in that case is that there should be no way for you during those 6 months to see the intermediate results. Another way would be to not limit the test by time, but instead define beforehand on the number of trials..... i.e. say "this test will end after 100 trials were done". If you go with the predefined number of trials approach, then you are not allowed to end the test before you complete the 100 trials, else the results will be invalid.


--------------------
I am arrogant and I can afford it because I deliver.
Go to the top of the page
+Quote Post

14 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th July 2014 - 08:09