IPB

Welcome Guest ( Log In | Register )

ABX testing vs. unconscious perception/decision, enjoyment, and so on, Split from “xiphmont—‘There is no point to 24/192’”/TID:93853 (TOS #5)
icstm
post Mar 16 2012, 12:31
Post #1





Group: Members
Posts: 121
Joined: 25-January 12
Member No.: 96698



SOME THOUGHTS

All this talk about skin “hearing” music, “feeling” the vibrations of the music etc raises for me a separate debate.

Whilst I completely agree that ABX is an excellent tool for spotting consciously detectable differences (ie respondents must be wilfully aware of the difference to make their choice) what about our unconscious decisioning? Just the other day their was a BBC documentary “out of control” which they were explaining how your unconscious mind is often calling the shots and influencing the decisions you make.

So we have 3 different types of tests:
1) Signal waveform analysis (as was used to check to see if JPlay was doing anything or FLAC v WAV using Audio DiffMaker
2) ABX audio testing (man in the middle) for checking for conscious discrimination of different samples
3) Enjoyment factor (or some other form of what some philosophers call qualia)


I am not saying to know that our skin reacts to sound or anything like that however I can believe that there is more to our enjoyment of music than just what our ears hear, for example I was at a classical concert yesterday and we slipped from the back to some seats in the front where we could feel the power of the music more in our bellies. Is this just due to the increased volume and would our ears have provided our brains with sufficient information if I had desensitised my stomach – I am not so sure. Now clearly in this example I was conscious of this effect, but there may be others I was not and yet still added to my overall enjoyment.

So even assuming that enjoyment is a wholly conscious act there could be unconscious inputs that the brain turns into conscious states. However these states could be impacted by the ABX test in 2 ways:
A. Not enough time is allowed for in playing the samples for these unconscious inputs to take effect
B. The act of enjoying a concert vs the act of trying to spot differences mean the brain is trying to do different things and thus doing the second might not require the same inputs as used in the first (I assume we are happy that the outputs of those 2 exercises are likely to be different and in a different form

Now bear in mind that for this thread I am in the 16/44 is sufficient for playback camp – I am still saying I am not fully satisfied that tests (1) and (2) are sufficient to provide complete proof. It is a bit like evolution (as some other poster mentioned). There is some proof for the theory, there are many wholes (for example in the fossil line) however there is no proof for any other theory...

This post has been edited by icstm: Mar 16 2012, 12:39
Go to the top of the page
+Quote Post
 
Start new topic
Replies
pdq
post Mar 16 2012, 13:32
Post #2





Group: Members
Posts: 3443
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



Don't assume any limitations to how ABX testing is applied. The only real limitation is that it needs to be applied double-blind, i.e. there can not be any means other than the effect being tested to differentiate A from B from X.

Your reference to conscious decisions isn't even a limitation. There is no reason that the subject even be aware that he is taking an ABX test if the observer can evaluate his reactions without them being verbalized. Similarly there is no requirement that the testing be within some time period. Testing could take place over months or years and still be just as valid.

Finally, I will point out what has been said many many times before. ABX testing cannot prove that there is no difference between A and B. It can only provide reasonable certainty that there is.
Go to the top of the page
+Quote Post
Porcus
post Mar 16 2012, 15:20
Post #3





Group: Members
Posts: 1957
Joined: 30-November 06
Member No.: 38207



QUOTE (pdq @ Mar 16 2012, 13:32) *
Don't assume any limitations to how ABX testing is applied. The only real limitation is that it needs to be applied double-blind,


The ABX test method and the double-blindness are distinct concepts. The former is a type of experiment, the latter is a method to address and mitigate a certain type of bias. You can have double-blind experiments that are not of the ABX type, and you can have ABX with or without double-blinding. Question is rather, which one(s) of these four combinations (ABX or not, double-blind or not) it makes sense to use.

If we are facing the issue of use of A/B tests which are prone to placebo-type bias, then a very modest request for improvement would be: Suggest the smallest possible blinded (i.e. 'blindable'!) modification to sighted A/B. The mitigation of placebo is an end, the double-blinding is a well-established means to that, and ABX is ... well, basically iterated comparison, so it passes as a small modification to AB. Problem solved.

(In addition to 'ABX in place of A/B' and 'double-blind', this forum also would expect a standard to statistical procedure to evaluate the scores before drawing inference from them. That is a third distinct concept!)


QUOTE (pdq @ Mar 16 2012, 13:32) *
i.e. there can not be any means other than the effect being tested to differentiate A from B from X.
[...]
Testing could take place over months or years and still be just as valid.


Ah, well, in principle. In practice, you could easily violate the ceteris paribus desideratum, and it would be hard to get sensible results. There is an entire branch of the statistical science dedicated to the design of experiment.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Mar 16 2012, 18:11
Post #4





Group: Members
Posts: 4285
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (Porcus @ Mar 16 2012, 10:20) *
QUOTE (pdq @ Mar 16 2012, 13:32) *
Don't assume any limitations to how ABX testing is applied. The only real limitation is that it needs to be applied double-blind,


The ABX test method and the double-blindness are distinct concepts. The former is a type of experiment, the latter is a method to address and mitigate a certain type of bias. You can have double-blind experiments that are not of the ABX type, and you can have ABX with or without double-blinding. Question is rather, which one(s) of these four combinations (ABX or not, double-blind or not) it makes sense to use.

If we are facing the issue of use of A/B tests which are prone to placebo-type bias, then a very modest request for improvement would be: Suggest the smallest possible blinded (i.e. 'blindable'!) modification to sighted A/B. The mitigation of placebo is an end, the double-blinding is a well-established means to that, and ABX is ... well, basically iterated comparison, so it passes as a small modification to AB. Problem solved.


The above paragraph pretty much describes our state of mind when we invented ABX.

Our first attempt involved creating a hidden list of As and Bs that were assigned randomly before the test and maintained throughout the test.

We'd play the alternatives in accordance with the hidden list and ask people to respond by writing down on a personal piece of paper first which one they liked the most, and later on whether they thought that List Item N was A or B. We made this change of direction because people complained that they thought they could hear a difference, but the alternatives were too much alike to create a preference.

This was found to be too hard because it put too much burden on our memory for the subtle details of what things sounded like.

Then we provided sources known to be A and B as references that the listener could compare to the unknown (which we started calling X) as many times as desired. That was ABX and it always seemed to be wonderfully easy compared to what went before it.

This post has been edited by Arnold B. Krueger: Mar 16 2012, 18:15
Go to the top of the page
+Quote Post

Posts in this topic
- icstm   ABX testing vs. unconscious perception/decision, enjoyment, and so on   Mar 16 2012, 12:31
- - dhromed   QUOTE (icstm @ Mar 16 2012, 12:31) I was ...   Mar 16 2012, 12:56
- - Speedskater   In a really good ABX test: First] The listener sh...   Mar 16 2012, 13:24
- - pdq   Don't assume any limitations to how ABX testin...   Mar 16 2012, 13:32
|- - Porcus   QUOTE (pdq @ Mar 16 2012, 13:32) Don...   Mar 16 2012, 15:20
|- - Arnold B. Krueger   QUOTE (Porcus @ Mar 16 2012, 10:20) QUOTE...   Mar 16 2012, 18:11
|- - Porcus   QUOTE (Arnold B. Krueger @ Mar 16 2012, 18...   Mar 16 2012, 19:20
- - icstm   QUOTE (dhromed @ Mar 16 2012, 11:56) QUOT...   Mar 16 2012, 14:10
|- - Arnold B. Krueger   QUOTE (icstm @ Mar 16 2012, 09:10) QUOTE ...   Mar 16 2012, 14:46
- - pdq   There are many ways that one could envision determ...   Mar 16 2012, 14:20
- - KMD   I agree that being engrosed in a listening experi...   Mar 16 2012, 14:25
|- - Arnold B. Krueger   QUOTE (KMD @ Mar 16 2012, 09:25) I agree ...   Mar 16 2012, 14:51
- - icstm   pdq - I agree that there should be a way to measu...   Mar 16 2012, 14:31
- - KMD   How about adding market discipline to an ABX test....   Mar 16 2012, 14:36
- - KMD   I wasn't refering to a live venue. By listeni...   Mar 16 2012, 15:00
- - pdq   I only mention testing over longer periods to coun...   Mar 16 2012, 15:33
- - drewfx   QUOTE (icstm @ Mar 16 2012, 06:31) SOME T...   Mar 16 2012, 16:52
|- - icstm   @drewfx I think you are missing what I am saying....   Mar 16 2012, 17:16
|- - drewfx   QUOTE (icstm @ Mar 16 2012, 11:16) I am s...   Mar 16 2012, 17:46
|- - saratoga   QUOTE (icstm @ Mar 16 2012, 11:16) As KMD...   Mar 16 2012, 21:02
- - KMD   icstm - yes an EEG filts in with my idea of how ...   Mar 16 2012, 17:22
- - googlebot   If your primary listening situation is sighted, an...   Mar 16 2012, 20:16
|- - Porcus   QUOTE (googlebot @ Mar 16 2012, 20:16) If...   Mar 18 2012, 15:04
|- - googlebot   QUOTE (Porcus @ Mar 18 2012, 15:04) I am ...   Mar 18 2012, 15:40
|- - splice   QUOTE (googlebot @ Mar 18 2012, 06:40) .....   Mar 19 2012, 00:17
|- - Arnold B. Krueger   QUOTE (splice @ Mar 18 2012, 19:17) QUOTE...   Mar 19 2012, 16:56
||- - Porcus   QUOTE (Arnold B. Krueger @ Mar 19 2012, 16...   Mar 19 2012, 19:04
|- - googlebot   QUOTE (splice @ Mar 19 2012, 00:17) Er......   Mar 19 2012, 20:04
|- - Arnold B. Krueger   QUOTE (googlebot @ Mar 19 2012, 15:04) QU...   Mar 20 2012, 14:38
- - icstm   BACK TO MY OP How does THIS page change things? It...   Mar 19 2012, 12:59
- - greynol   A search of the forum should have revealed that th...   Mar 19 2012, 13:03
- - icstm   ah, did not realise that it was the much discredit...   Mar 19 2012, 14:55
|- - DonP   QUOTE (icstm @ Mar 19 2012, 09:55) Has an...   Mar 19 2012, 15:09
- - icstm   yeah, I was thinking about that too. Plus they are...   Mar 19 2012, 16:09
- - icstm   A PLL is to sync to signals to the same clock, whe...   Mar 20 2012, 16:48
- - pdq   The amount of filtering that you can apply to the ...   Mar 20 2012, 17:37
- - Arnold B. Krueger   QUOTE (icstm @ Mar 16 2012, 07:31) SOME T...   Mar 20 2012, 18:09
- - icstm   If you are forgetting that you are doing an ABX, t...   Mar 22 2012, 12:36
|- - Arnold B. Krueger   QUOTE (icstm @ Mar 22 2012, 07:36) If you...   Mar 23 2012, 00:08
- - greynol   http://www.hydrogenaudio.org/forums/index....st...   Mar 22 2012, 14:31
|- - icstm   @greynol You've lost me there, Arnold was prov...   Mar 22 2012, 17:04
|- - drewfx   QUOTE (icstm @ Mar 22 2012, 11:04) Drewfi...   Mar 22 2012, 18:05
|- - icstm   QUOTE (drewfx @ Mar 22 2012, 17:05) If yo...   Mar 23 2012, 12:44
|- - Porcus   Could a signal just above the hearing threshold ca...   Mar 23 2012, 17:29
- - greynol   I guess I'd like to see you answer drewfx...   Mar 22 2012, 17:59
|- - Arnold B. Krueger   QUOTE (greynol @ Mar 22 2012, 12:59) I gu...   Mar 23 2012, 00:12
|- - dhromed   QUOTE (Arnold B. Krueger @ Mar 23 2012, 00...   Mar 23 2012, 00:46
|- - drewfx   QUOTE (Arnold B. Krueger @ Mar 22 2012, 18...   Mar 23 2012, 02:06
|- - drewfx   QUOTE (Arnold B. Krueger @ Mar 22 2012, 18...   Mar 23 2012, 21:10
- - greynol   To take this further, someone investing time and/o...   Mar 22 2012, 18:23
- - greynol   Here we go again with the flying spaghetti monster...   Mar 23 2012, 01:26
- - greynol   Because of my masochistic tendencies, I had to go ...   Mar 23 2012, 06:03
|- - 2Bdecided   QUOTE (greynol @ Mar 23 2012, 05:03) Spoi...   Mar 23 2012, 10:30
- - icstm   @greynol There was a post in that old thred you p...   Mar 23 2012, 12:51
- - greynol   Are we expected to believe the more consciously at...   Mar 23 2012, 17:13
- - icstm   I agree, but nor have I seen evidence for any test...   Mar 26 2012, 14:42
- - greynol   Should I start binning appeals to the flying spagh...   Mar 26 2012, 15:01
- - icstm   but we have satellites and telescopes that would h...   Mar 27 2012, 10:41
|- - DonP   QUOTE (icstm @ Mar 27 2012, 05:41) We als...   Mar 27 2012, 14:06
|- - greynol   QUOTE (icstm @ Mar 27 2012, 02:41) but we...   Mar 27 2012, 19:44
- - 2Bdecided   This "subconscious perception" is a red ...   Mar 27 2012, 14:14
- - googlebot   That placebo can cause a significant effect on the...   Mar 27 2012, 19:19
|- - greynol   QUOTE (googlebot @ Mar 27 2012, 11:19) So...   Mar 27 2012, 19:29
|- - googlebot   I have stopped arguing with audiophiles in my soci...   Mar 27 2012, 20:03
||- - JJZolx   QUOTE (googlebot @ Mar 27 2012, 12:03) Ju...   Mar 28 2012, 02:30
|- - Nessuno   QUOTE (greynol @ Mar 27 2012, 19:29) Hope...   Mar 28 2012, 09:39
|- - Porcus   QUOTE (greynol @ Mar 27 2012, 20:29) I wa...   Mar 28 2012, 10:15
- - greynol   QUOTE (googlebot @ Mar 27 2012, 12:03) No...   Mar 27 2012, 20:21
- - Woodinville   A blog I wrote on the psychology behind this issue...   Mar 27 2012, 23:28
- - icstm   QUOTE (DonP @ Mar 27 2012, 13:06) QUOTE (...   Mar 28 2012, 11:06
|- - Woodinville   QUOTE (icstm @ Mar 28 2012, 03:06) [We ha...   Mar 28 2012, 20:39
- - knutinh   If it cannot be registered, cannot make reliable p...   Mar 28 2012, 11:33
- - icstm   QUOTE (Woodinville @ Mar 28 2012, 19:39) ...   Mar 29 2012, 10:15
|- - 2Bdecided   QUOTE (icstm @ Mar 29 2012, 09:15) I mean...   Mar 29 2012, 11:10
||- - icstm   QUOTE (2Bdecided @ Mar 29 2012, 10:10) QU...   Mar 29 2012, 11:55
||- - Woodinville   QUOTE (2Bdecided @ Mar 29 2012, 03:10) It...   Mar 30 2012, 02:09
|- - Woodinville   QUOTE (icstm @ Mar 29 2012, 02:15) QUOTE ...   Mar 30 2012, 02:06
|- - icstm   QUOTE (Woodinville @ Mar 30 2012, 01:06) ...   Mar 30 2012, 10:25
|- - Woodinville   QUOTE (icstm @ Mar 30 2012, 02:25) QUOTE ...   Mar 30 2012, 21:09
|- - Kees de Visser   QUOTE (icstm @ Mar 30 2012, 11:25) I did ...   Apr 2 2012, 11:40
- - icstm   @Woodinville You are right, I am not well versed i...   Apr 2 2012, 10:23
|- - greynol   QUOTE (icstm @ Apr 2 2012, 02:23) From wh...   Apr 2 2012, 17:29
|- - Woodinville   QUOTE (greynol @ Apr 2 2012, 09:29) QUOTE...   Apr 3 2012, 09:20
||- - Paulhoff   QUOTE (Woodinville @ Apr 3 2012, 04:20) I...   Apr 4 2012, 00:35
|- - icstm   QUOTE (greynol @ Apr 2 2012, 16:29) The p...   Apr 3 2012, 10:44
|- - Woodinville   QUOTE (icstm @ Apr 3 2012, 02:44) The fac...   Apr 3 2012, 23:32
|- - icstm   @ 'Woodinville' Actually I am completely...   Apr 4 2012, 10:24
|- - Woodinville   QUOTE (icstm @ Apr 4 2012, 02:24) However...   Apr 4 2012, 12:03
||- - icstm   QUOTE (Woodinville @ Apr 4 2012, 11:03) U...   Apr 5 2012, 11:32
||- - 2Bdecided   QUOTE (icstm @ Apr 5 2012, 11:32) But I a...   Apr 5 2012, 11:47
||- - Arnold B. Krueger   QUOTE (icstm @ Apr 5 2012, 06:32) QUOTE (...   Apr 6 2012, 16:44
|- - Arnold B. Krueger   QUOTE (icstm @ Apr 4 2012, 05:24) However...   Apr 4 2012, 13:01
|- - Paulhoff   QUOTE (Arnold B. Krueger @ Apr 4 2012, 08...   Apr 4 2012, 16:36
|- - Arnold B. Krueger   QUOTE (Paulhoff @ Apr 4 2012, 11:36) QUOT...   Apr 6 2012, 16:32
- - greynol   I wouldn't necessarily assume that discussions...   Apr 3 2012, 15:27
- - icstm   Hope everyone had a good Easter break. Reading tho...   Apr 10 2012, 11:19


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th November 2014 - 07:36