IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3 >  
Reply to this topicStart new topic
Positive ABX test of loudspeaker cables
DonP
post Jul 20 2011, 11:31
Post #26





Group: Members (Donating)
Posts: 1471
Joined: 11-February 03
From: Vermont
Member No.: 4955



QUOTE (hybris @ Jul 20 2011, 04:19) *
If he was pressed to explain what the difference was, he'd say that one of the cables had a more spacious sound, and somewhat crisper in some parts of the track.


And even with that, maybe the better one is the cheaper one.

QUOTE (RobWansbeck)
This is at odds with the advertising blurb, e.g.

“ The high tolerance conductor spacing used in this cable means that musical performance reaches new heights. For detail, resolution and transparency Baldur will surprise and delight you. “

One would have hoped that a cable that surprises and delights would have been more detectable.


Their literature also emphasizes the Baldur being good value for the money. You know, not that snake oil crazy expensive stuff.
Go to the top of the page
+Quote Post
ojdo
post Jul 20 2011, 13:03
Post #27





Group: Members
Posts: 894
Joined: 18-June 06
From: Germany
Member No.: 31980



QUOTE (hellokeith @ Jul 20 2011, 10:13) *
A bit simplified, but: p=.5, n=30, x=22 --> P(x) = .0055

Calculation question: I got 0.8% for the probability of 22 or more hits in case of guessing (p=0.5). I think you calculated the probability for obtaining exactly 22 hits. What is correct for p-level calculation?
CODE
octave-3.2.4.exe:15> 1 - binocdf(21, 30, .5)
ans =  0.0080624


--------------------
http://freemusi.cc/
Go to the top of the page
+Quote Post
2Bdecided
post Jul 20 2011, 13:24
Post #28


ReplayGain developer


Group: Developer
Posts: 5181
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (RobWansbeck @ Jul 20 2011, 00:50) *
Although the overall result was statistically significant none of the individual listeners could achieve a statistically significant result. This is at odds with the advertising blurb, e.g.

“ The high tolerance conductor spacing used in this cable means that musical performance reaches new heights. For detail, resolution and transparency Baldur will surprise and delight you. “

One would have hoped that a cable that surprises and delights would have been more detectable.
smile.gif

But honest marketing wouldn't sell cables.

Isn't "honest marketing" an oxymoron anyway?

Cheers,
David.
Go to the top of the page
+Quote Post
Hansen
post Jul 20 2011, 15:17
Post #29





Group: Members
Posts: 29
Joined: 25-September 10
Member No.: 84131



Being Norwegian, I've followed the discussion on AVForum with some interest, and to me it seems obvious that

1, the changes of cables were performed by a person being present at the same time and visible to the listeners.

2, the changes were not random, but decided by the 'changers' on a 50/50 basis.

Both these circumstances make the test useless..

The people criticizing the 'test' seems focused on the mathematical chances of guessing — but it's actually a matter of psychology and non-verbal communication. I'm actually surprised the result was not higher than 70-80%
Go to the top of the page
+Quote Post
hybris
post Jul 20 2011, 22:07
Post #30





Group: Members
Posts: 97
Joined: 24-March 09
From: Oslo
Member No.: 68343



QUOTE (Hansen @ Jul 20 2011, 16:17) *
Being Norwegian, I've followed the discussion on AVForum with some interest, and to me it seems obvious that

1, the changes of cables were performed by a person being present at the same time and visible to the listeners.

2, the changes were not random, but decided by the 'changers' on a 50/50 basis.

Both these circumstances make the test useless..

The people criticizing the 'test' seems focused on the mathematical chances of guessing — but it's actually a matter of psychology and non-verbal communication. I'm actually surprised the result was not higher than 70-80%


The fact that it wasn't a double blind test weakens the test, but as far as I know regular blind testing isn't totally useless. I would assume the added value of adding double blind depends on the nature of the test.


Bias in blind (not double blind) testing may occur for example if a doctor knows that the patient is getting placebo, and thus may without meaning to do so send out negative signals to the patient or some kind of body language indicating that the doctor doesn't believe that the pills will work. This may then be picked up by the patient, and affect the test.

I don't see how this issue is as relevant (I'm not saying it is totally irrelevant) in a test as the one discussed here. The listener isn't told to detect the most expensive cable, simply if X is A or B, and it is equally important that he guesses correctly if it is the cheap or expensive cable he is trying to identify.

In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?



--------------------
Thorbjorn
Go to the top of the page
+Quote Post
krabapple
post Jul 20 2011, 22:09
Post #31





Group: Members
Posts: 2334
Joined: 18-December 03
Member No.: 10538



QUOTE (ojdo @ Jul 20 2011, 08:03) *
QUOTE (hellokeith @ Jul 20 2011, 10:13) *
A bit simplified, but: p=.5, n=30, x=22 --> P(x) = .0055

Calculation question: I got 0.8% for the probability of 22 or more hits in case of guessing (p=0.5). I think you calculated the probability for obtaining exactly 22 hits. What is correct for p-level calculation?
CODE
octave-3.2.4.exe:15> 1 - binocdf(21, 30, .5)
ans =  0.0080624




?

Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)

Go to the top of the page
+Quote Post
DonP
post Jul 20 2011, 22:31
Post #32





Group: Members (Donating)
Posts: 1471
Joined: 11-February 03
From: Vermont
Member No.: 4955



QUOTE (krabapple @ Jul 20 2011, 17:09) *
Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)


It's easier to lean towards more trials when you've got some automation and you can pop them off on your computer in a few minutes.

If it involves arranging for a facility, people to do the switching, leaving and entering rooms, and (as recounted here) it takes all day to do 10 trials x 3 listeners maybe take what you can get.
Go to the top of the page
+Quote Post
ojdo
post Jul 21 2011, 07:57
Post #33





Group: Members
Posts: 894
Joined: 18-June 06
From: Germany
Member No.: 31980



QUOTE (krabapple @ Jul 20 2011, 23:09) *
?

Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)

Ok, bad wording on my side. I used "p" both for the probability for naming the right cable per trial (null hypothesis guessing: p=0.5) as well as for the outcome of the probability that the 22/30 result would be obtained. When judging whether if falls into the category "statistically significant" or not, which probabilty is to be calculated?

a) The probability for exactly 22 hits by chance?
b) The probability for 22 or more hits by chance?


--------------------
http://freemusi.cc/
Go to the top of the page
+Quote Post
krabapple
post Jul 21 2011, 17:40
Post #34





Group: Members
Posts: 2334
Joined: 18-December 03
Member No.: 10538



QUOTE (ojdo @ Jul 21 2011, 01:57) *
QUOTE (krabapple @ Jul 20 2011, 23:09) *
?

Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)

Ok, bad wording on my side. I used "p" both for the probability for naming the right cable per trial (null hypothesis guessing: p=0.5) as well as for the outcome of the probability that the 22/30 result would be obtained. When judging whether if falls into the category "statistically significant" or not, which probabilty is to be calculated?

a) The probability for exactly 22 hits by chance?
b) The probability for 22 or more hits by chance?


try plugging the numbers in here
http://stattrek.com/tables/binomial.aspx
Go to the top of the page
+Quote Post
SCOTU
post Jul 21 2011, 17:57
Post #35





Group: Members
Posts: 118
Joined: 9-July 10
Member No.: 82156



QUOTE (hybris @ Jul 20 2011, 17:07) *
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?


I've always been under the impression that double blind tests are necessary for experiments such as these due to exactly the fact that people are good at picking up on subconscious cues of those around them. However, I've just been going on the word of mouth that people have told me this. I would be rather interested to see experimental results that indicate that Double Blind Experiments are needed.
Go to the top of the page
+Quote Post
hybris
post Jul 21 2011, 19:47
Post #36





Group: Members
Posts: 97
Joined: 24-March 09
From: Oslo
Member No.: 68343



QUOTE (SCOTU @ Jul 21 2011, 18:57) *
QUOTE (hybris @ Jul 20 2011, 17:07) *
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?


I've always been under the impression that double blind tests are necessary for experiments such as these due to exactly the fact that people are good at picking up on subconscious cues of those around them. However, I've just been going on the word of mouth that people have told me this. I would be rather interested to see experimental results that indicate that Double Blind Experiments are needed.


I agree. It's one thing if the cable changer stands in the room and asks the guy if he thinks this cable sounds better than the previous one, while knowing that it is the expensive cable that is playing. But in this particular context (identify if X is A or B), I'm not sure how the cable changer should be able to subcounsciously help the tester.


--------------------
Thorbjorn
Go to the top of the page
+Quote Post
DonP
post Jul 21 2011, 21:32
Post #37





Group: Members (Donating)
Posts: 1471
Joined: 11-February 03
From: Vermont
Member No.: 4955



QUOTE (hybris @ Jul 21 2011, 14:47) *
I agree. It's one thing if the cable changer stands in the room and asks the guy if he thinks this cable sounds better than the previous one, while knowing that it is the expensive cable that is playing. But in this particular context (identify if X is A or B), I'm not sure how the cable changer should be able to subcounsciously help the tester.


We weren't there to see what interaction was possible, but as an example a more upbeat tone of voice or expression when he announces that the the high price cable is ready, whether it is A, B, or X. Maybe he sways to the music a bit when the high price cable is playing.

When the forum was discussing the bases they had to cover so the test would be unassailable I'm surprised double-blind didn't come up, since it is generally a standard for ABX testing. For that matter, so is random selection of X, but that part isn't so much in the public consciousness.

This post has been edited by DonP: Jul 21 2011, 21:34
Go to the top of the page
+Quote Post
hybris
post Jul 21 2011, 22:34
Post #38





Group: Members
Posts: 97
Joined: 24-March 09
From: Oslo
Member No.: 68343



QUOTE (DonP @ Jul 21 2011, 22:32) *
We weren't there to see what interaction was possible, but as an example a more upbeat tone of voice or expression when he announces that the the high price cable is ready, whether it is A, B, or X. Maybe he sways to the music a bit when the high price cable is playing.

When the forum was discussing the bases they had to cover so the test would be unassailable I'm surprised double-blind didn't come up, since it is generally a standard for ABX testing. For that matter, so is random selection of X, but that part isn't so much in the public consciousness.


But there's no immediate relevance as to wether the expensive or cheap cable is playing. The test isn't about figuring out which cable is best, it is as about successfully identifying X each time, regardless of wether X is the cheap or the expensive cable. I assume the guy could send "expensive signals" to the listeners when the expensive cable is playing, and if the listener gets this on B, but not A or X - he could assume that X is A. But I must admit I have my doubts as to how much this would affect this test in practice.

When it comes to swaying to the music: The person that changed the cable was not in the room while the listening was conducted. He exits and enters the room between each change. But I guess he may be walking faster in anticipation each time he has changed to the expensive cable, we have no real way of knowing how all this works. Which is the point of double blind in the first place I guess.

My point is just that it may be slightly unfair / rushed to say that the test / result is totally useless because it wasn't double blind. smile.gif


--------------------
Thorbjorn
Go to the top of the page
+Quote Post
Notat
post Jul 22 2011, 03:14
Post #39





Group: Members
Posts: 581
Joined: 17-August 09
Member No.: 72373



It looks like reasonable care was taken in testing. The simplest explanation appears to be that there are audible differences between the cables. I suggest someone measure the the frequency response and attenuation of the cables while they are connected to the speakers used in the test. It's not a difficult test to do. You can even take a crack with a test CD and a digital volt meter if you don't have real audio test equipment.

The expensive cables are a peculiar geometry and it is quite possible they will affect the sound. Not necessarily better but audibly different.
Go to the top of the page
+Quote Post
Speedskater
post Jul 22 2011, 21:49
Post #40





Group: Members
Posts: 139
Joined: 12-December 07
From: Cleveland, OH
Member No.: 49500



In another audio forum, an older thread about DBT'ing just today had a link to an interconnect cable blind test:

http://www.nordost.com/pdf/hifiplus_issue34.pdf

To me it seems rather lacking in details about just how blind of a test it was.


--------------------
Kevin
Go to the top of the page
+Quote Post
Speedskater
post Jul 22 2011, 22:20
Post #41





Group: Members
Posts: 139
Joined: 12-December 07
From: Cleveland, OH
Member No.: 49500



QUOTE (Speedskater @ Jul 22 2011, 16:49) *
In another audio forum, an older thread about DBT'ing just today had a link to an interconnect cable blind test:
http://www.nordost.com/pdf/hifiplus_issue34.pdf
To me it seems rather lacking in details about just how blind of a test it was.


After further review.
"Hi Fi Plus" magazine issue #34 has a 2004 date and I don't see this test listed on their web page.


--------------------
Kevin
Go to the top of the page
+Quote Post
andy o
post Jul 22 2011, 22:31
Post #42





Group: Members
Posts: 1333
Joined: 14-April 09
Member No.: 68950



QUOTE (hybris @ Jul 19 2011, 13:43) *
What are the obvious shortcomings, besides the fact that it wasn't double blind?

The more obvious one to me, is that it wasn't random. Furthermore, I'm wondering if these two are true:

QUOTE (Hansen @ Jul 20 2011, 07:17) *
1, the changes of cables were performed by a person being present at the same time and visible to the listeners.

2, the changes were not random, but decided by the 'changers' on a 50/50 basis.


So, you are saying that #1 is not true at all, from your posts, right?

#2, it was decided on a 50/50 basis? Which means that not only it wasn't random, but also the "changers" made sure there was equal number of A's and B's?
Go to the top of the page
+Quote Post
hybris
post Jul 23 2011, 08:18
Post #43





Group: Members
Posts: 97
Joined: 24-March 09
From: Oslo
Member No.: 68343



1. The listener was not removed from the room while the change was taking place, so the listener could see the person who did the change. But the person that changed cables left the room before the listening was conducted, and the cable change was done behind a curtain, so it was of course not visible for the listener.

2. The changes was decided by the "changer" as the test went along, but X was the cheap and expensive cable 50% of the time. This is probably another weakness in the test, but as far as I know, the listeners were not aware of this fact during the test.

This post has been edited by db1989: Jul 23 2011, 09:12
Reason for edit: no need to quote a post directly above; please take 2 s to delete the auto-quote


--------------------
Thorbjorn
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Jul 26 2011, 13:16
Post #44





Group: Members
Posts: 3987
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (hybris @ Jul 20 2011, 17:07) *
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?


The means need not be precise, just effective!

Our need for DBTs becomes clear if one considers the story of "Clever Hans" a horse that "did arithmetic" but was actually sensing the right answers from his handlers.

A number of means by which Clerver Hans' handlers communicated the right answers to the horse have been documented, if memory serves. You might want to study up on this.
Go to the top of the page
+Quote Post
db1989
post Jul 26 2011, 13:48
Post #45





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



Now that’s an interesting study; thanks!

A few quotes from Hans’ Wikipedia article:
QUOTE
the horse got the right answer only when the questioner knew what the answer was, and the horse could see the questioner. He observed that when von Osten knew the answers to the questions, Hans got 89 percent of the answers correct, but when von Osten did not know the answers to the questions, Hans only answered six percent of the questions correctly.

Pfungst then proceeded to examine the behaviour of the questioner in detail, and showed that as the horse's taps approached the right answer, the questioner's posture and facial expression changed in ways that were consistent with an increase in tension, which was released when the horse made the final, correct tap. This provided a cue that the horse could use to tell it to stop tapping.

Pfungst made an extremely significant observation. After he had become adept at giving Hans performances himself, and fully aware of the subtle cues which made them possible, he discovered that he would produce these cues involuntarily regardless of whether he wished to exhibit or suppress them. Recognition of this striking phenomenon has had a large effect on experimental design and methodology for all experiments whatsoever involving sentient subjects (including humans).
Go to the top of the page
+Quote Post
hybris
post Jul 26 2011, 18:18
Post #46





Group: Members
Posts: 97
Joined: 24-March 09
From: Oslo
Member No.: 68343



Interesting study. smile.gif

Hopefully the test will be repeated as double blind later this year.


--------------------
Thorbjorn
Go to the top of the page
+Quote Post
greynol
post Jul 26 2011, 18:24
Post #47





Group: Super Moderator
Posts: 10048
Joined: 1-April 04
From: San Francisco
Member No.: 13167



...with each and every trial chosen randomly by coin toss and hopefully a larger number of participants and trials.

Was there anything else called into question?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
hybris
post Jul 26 2011, 19:10
Post #48





Group: Members
Posts: 97
Joined: 24-March 09
From: Oslo
Member No.: 68343



QUOTE (greynol @ Jul 26 2011, 19:24) *
...with each and every trial chosen randomly by coin toss and hopefully a larger number of participants and trials.

Was there anything else called into question?


A larger number of participants and trials may prove difficult as it apparently is quite time consuming to conduct the tests, but time will tell.

The main issues that was suggested changed for the next round was:

a) ensure double blind testing

b) selecting X randomly (coin toss or similar)

c) determine wether level matching between cables are required (can difference in resistance in the cables produce audible differences in volume?)

d) Switch cables both on the loudspeaker and amplifier end. During the previous test, the cables were only changed on the loudspeaker end, as the amps (AW180) has a double set of speaker terminals. So theoretically the test may have proved that speaker terminal A and speaker terminal B on the AW180s doesn't sound the same (I don't know how plausible that is in practice).


--------------------
Thorbjorn
Go to the top of the page
+Quote Post
MichaelW
post Jul 26 2011, 21:09
Post #49





Group: Members
Posts: 631
Joined: 15-March 07
Member No.: 41501



And to repeat @Notat's suggestion, it would surely be (time)cost-effective to do some measurements on the expensive cables, to see if they are colouring the sound?
Go to the top of the page
+Quote Post
Notat
post Jul 27 2011, 14:58
Post #50





Group: Members
Posts: 581
Joined: 17-August 09
Member No.: 72373



QUOTE (Arnold B. Krueger @ Jul 26 2011, 06:16) *
Our need for DBTs becomes clear if one considers the story of "Clever Hans" a horse that "did arithmetic" but was actually sensing the right answers from his handlers.

The handler and the horse never see one another in this test. I know that it can be subtle but I don't see a mechanism for conveying information to the participant.

Please do some measurements first. That will be much easier and objective than doing more listening. It is not difficult to build a cable that intentionally colors sound. Demonstrating that such a cable sounds different is not a great feat. I suspect that's what you've done here.
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th October 2014 - 23:06