IPB

Welcome Guest ( Log In | Register )

Biophysics, Limitations of Shannon and Issues with ABX Testing, Split from Topic ID: 57406 (TOS #5)
theorist1
post Jun 14 2013, 04:32
Post #1





Group: Members
Posts: 4
Joined: 14-June 13
Member No.: 108646



As a scientist (biophysics) and an audiophile, I have my feet in both worlds, and have thus enjoyed this discussion. Here are some thoughts:

I. Open Access.

Several have written it’s unfortunate that this paper, which is of general public interest, is not available to those without an AES subscription (that includes myself). Agreed; and note that the landscape for scientific publication has been changing: researchers are increasingly realizing the value of “open access” papers – ones available freely, without a subscription. Some scientific journals have gone entirely open access, while others (for an additional fee) offer the option of open access publication of your paper. Let me suggest that, if AES doesn’t offer this option, it should, and that those publishing papers of broad interest should endeavor (if financially possible) to avail themselves of this.


II. ABX testing.

I assume from the discussion that this paper makes use of sequential ABX testing (by “sequential” I mean: present A, then B, then X, in sequence). I’d like to see basic research done into sequential ABX testing itself. In particular, sequential ABX testing (double-blinding will be assumed throughout this discussion) is highly regarded for assessing perceptual discrimination, because (with proper statistical analysis) it eliminates false positives – subjects can’t “pass” (i.e., achieve successful discrimination in) a sequential ABX test unless they truly can reliably distinguish A from B. But what about the converse? Achieving accurate discrimination in sequential ABX testing is hard, because it requires accurate memory of both A and B when judging X. Therefore I’d like to offer the hypothesis that subjects can “fail” sequential ABX testing even when they can successfully discriminate between A and B, and suggest a way of testing this. Essentially, one would use simultaneous ABX testing to eliminate the memory requirement, and then compare the results to those for sequential ABX testing. I can’t think of how you’d do simultaneous ABX testing with hearing, but it could certainly be done with vision (using color tiles). The experiment would be divided into three phases:

1) Find the smallest color difference that can be reliably distinguished during simultaneous ABX testing (present all three tiles – A, B, and X – simultaneously, so that the subjects can line them up next to each other for comparison).
2) Repeat the experiment, using that same color difference, with sequential ABX testing.
3) If the subjects fail phase 2 (thus confirming my hypothesis), find the smallest color difference that can be reliably distinguished using sequential ABX testing, and compare to the results of phase 1.

If it is found that subjects indeed fail to reliably distinguish colors when presented sequentially that they can reliably distinguish when presented simultaneously, this will suggest that sequential ABX testing may not be a good method for assessing our perceptual limits (i.e., for determining transparency) -- and not just for vision, but possibly for hearing as well.


III. High-resolution audio and the Shannon sampling theorem.

Here I’ll apologize in advance if my questions are na´ve – I’m no expert at signal processing. Anyways:

Those who are dismissive of high-res audio based on theory alone typically cite the Shannon sampling theorem (typically misattributed to Nyquist), noting correctly that we can’t hear over 20 kHz (if that), and that we only need >40 kHz sampling to accurately reproduce this. But 2 x max. frequency is not the theorem’s only requirement. It’s my understanding that it also assumes an infinite signal, perfect sampling, and perfect interpolation. I’ve never seen any of these assumptions discussed in this context, so I’d like to ask how much practical effect these requirements would have on Redbook (16 bit/44.1 kHz) vs. high-res conversion:

1) Infinite signal. I assume we can effectively satisfy this with signal length >> 1/frequency, and that this is thus a non-issue.

2) Perfect sampling. Clearly, sampling need not be perfect, but simply close enough to perfect to be transparent in amplitude and time. Timing errors lead to jitter (right?). So (and this is an engineering question): what’s the relationship between sampling rate and how easy it is to eliminate audible jitter errors?

3) Perfect interpolation. Another engineering question: Naively, unless implementing near-perfect (i.e., transparent) interpolation is trivial, I would think it would be easier to achieve transparent interpolation with a higher sampling rate, because the points are more closely spaced. Is transparent interpolation so easily achievable that the sampling rate has no practical effect?


IV. The effect of mastering.

Many have mentioned that the reason high-res disks do indeed sound better than CDs of the same performance is that they are mastered differently – as labors of love, and without the usual commercial pressures to alter the sound. Given this, I think it would be a pubic service if someone could produce a Blu-Ray disk corresponding to the songs tested in this study, containing both the high-res and Redbook versions of each, and make it available for sale. That way people could easily experiment for themselves.


V. General thoughts.

I think the reason for the continued controversy about digital audio performance is that we don’t completely understand the biophysics of human hearing (which is why it continues to be an active area of research). If we did, we would know, a priori, what constitutes a complete specification set sufficient to determine transparency, and thus could engineer transparent electronic gear (I say electronic because I am excluding transducers) without listening to it. To the best of my understanding, this is not yet the case, since the errors that our auditory system is capable of detecting can be extraordinarily subtle, and what would constitute a complete set of scalar specifications sufficient to ensure transparency thus remains an open question.

This post has been edited by greynol: Jun 14 2013, 14:02
Reason for edit: Added link to original discussion from which this one was split for being off-topic.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
pdq
post Jun 14 2013, 05:17
Post #2





Group: Members
Posts: 3450
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



I don't know what you mean by "sequential" ABX testing. Any ABX test that I am aware of allows the subject to listen to any of the three samples, in any order, as many times as he/she pleases until ready to make a choice.

Also, you cannot "eliminate" false positives just as you cannot eliminate false negatives. You can only make the null hypothesis statistically improbable.
Go to the top of the page
+Quote Post
theorist1
post Jun 14 2013, 05:34
Post #3





Group: Members
Posts: 4
Joined: 14-June 13
Member No.: 108646



QUOTE (pdq @ Jun 13 2013, 20:17) *
I don't know what you mean by "sequential" ABX testing. Any ABX test that I am aware of allows the subject to listen to any of the three samples, in any order, as many times as he/she pleases until ready to make a choice.

Also, you cannot "eliminate" false positives just as you cannot eliminate false negatives. You can only make the null hypothesis statistically improbable.


The point is that it's sequential regardless of the order, and thus relies on memory -- you can't listen to all three tracks simultaneously. If you reread that section of my post carefully I think you'll see what I'm getting at.

And yes, I could (and probably should) have written that sentence less casually and said "reduce the probability of a false positive (where a false positive is defined in statistics as 'the incorrect rejection of a true null hypothesis'*) to a very low level" instead of "eliminate," but the difference between those two statements has entirely no bearing on any of the arguments I've made.

*source: http://en.wikipedia.org/wiki/Type_I_and_type_II_errors

This post has been edited by theorist1: Jun 14 2013, 05:54
Go to the top of the page
+Quote Post
greynol
post Jun 14 2013, 06:15
Post #4





Group: Super Moderator
Posts: 10338
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (theorist1 @ Jun 13 2013, 21:34) *
The point is that it's sequential regardless of the order, and thus relies on memory

Memory is always required in order to asses audible differences; not just for testing.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post

Posts in this topic
- theorist1   Biophysics, Limitations of Shannon and Issues with ABX Testing   Jun 14 2013, 04:32
- - pdq   I don't know what you mean by "sequential...   Jun 14 2013, 05:17
|- - theorist1   QUOTE (pdq @ Jun 13 2013, 20:17) I don...   Jun 14 2013, 05:34
|- - greynol   QUOTE (theorist1 @ Jun 13 2013, 21:34) Th...   Jun 14 2013, 06:15
||- - 2Bdecided   QUOTE (greynol @ Jun 14 2013, 06:15) QUOT...   Jun 14 2013, 09:38
||- - Martel   QUOTE (2Bdecided @ Jun 14 2013, 10:38) Ex...   Aug 6 2013, 17:20
|- - Woodinville   QUOTE (theorist1 @ Jun 13 2013, 20:34) QU...   Dec 9 2014, 22:26
- - Kees de Visser   QUOTE (theorist1 @ Jun 14 2013, 05:32) (b...   Jun 14 2013, 10:33
- - greynol   I hate to do this, since drawing allusions to sigh...   Jun 14 2013, 13:55
- - drewfx   QUOTE (theorist1 @ Jun 13 2013, 23:32) Ac...   Jun 14 2013, 17:54
|- - theorist1   Thanks for all your comments! QUOTE (greynol ...   Jun 15 2013, 03:11
|- - theorist1   QUOTE ([JAZ] @ Jun 14 2013, 09:32...   Jun 15 2013, 03:12
||- - [JAZ]   QUOTE (theorist1 @ Jun 15 2013, 04:12) So...   Jun 15 2013, 16:43
||- - greynol   QUOTE (theorist1 @ Jun 15 2013, 04:12) So...   Jun 17 2013, 07:00
|- - krabapple   QUOTE (theorist1 @ Jun 14 2013, 22:11) ly...   Jun 20 2013, 11:17
- - [JAZ]   I will try to address the theory and reasoning of ...   Jun 14 2013, 18:32
|- - pdq   QUOTE ([JAZ] @ Jun 14 2013, 13:32...   Jun 14 2013, 18:44
||- - splice   QUOTE (pdq @ Jun 14 2013, 09:44) ... The ...   Jun 15 2013, 00:10
|- - phofman   QUOTE ([JAZ] @ Jun 14 2013, 19:32...   Jun 16 2013, 19:49
- - Woodinville   QUOTE (theorist1 @ Jun 13 2013, 20:32) If...   Jun 15 2013, 07:16
- - Porcus   There is a sampling/interpolation/hearing issue th...   Jun 15 2013, 18:38
|- - Woodinville   QUOTE (Porcus @ Jun 15 2013, 10:38) Anywa...   Jun 16 2013, 07:34
|- - Porcus   QUOTE (Woodinville @ Jun 16 2013, 08:34) ...   Jun 17 2013, 10:53
|- - Woodinville   QUOTE (Porcus @ Jun 17 2013, 02:53) A 19 ...   Jun 18 2013, 03:10
|- - Porcus   QUOTE (Woodinville @ Jun 18 2013, 04:10) ...   Jun 18 2013, 10:41
|- - knutinh   QUOTE (Porcus @ Jun 18 2013, 10:41) Now i...   Jun 18 2013, 13:01
||- - Porcus   QUOTE (knutinh @ Jun 18 2013, 14:01) It a...   Jun 18 2013, 23:26
||- - knutinh   QUOTE (Porcus @ Jun 18 2013, 23:26) QUOTE...   Jun 19 2013, 06:24
|- - Woodinville   QUOTE (Porcus @ Jun 18 2013, 02:41) Now i...   Jun 20 2013, 06:26
- - knutinh   Scientific curiosity aside: If it was found that s...   Jun 17 2013, 07:44
|- - Porcus   QUOTE (knutinh @ Jun 17 2013, 08:44) If [...   Jun 17 2013, 09:56
|- - knutinh   QUOTE (Porcus @ Jun 17 2013, 09:56) Long-...   Jun 18 2013, 10:19
- - antz   QUOTE (theorist1 @ Jun 14 2013, 04:32) If...   Aug 4 2013, 13:46
|- - greynol   QUOTE (theorist1 @ Jun 14 2013, 04:32) If...   Aug 4 2013, 18:31
|- - Woodinville   QUOTE (antz @ Aug 4 2013, 05:46) The fact...   Aug 4 2013, 21:42
|- - antz   QUOTE (Woodinville @ Aug 4 2013, 21:42) Q...   Aug 4 2013, 22:14
- - Light-Fire   QUOTE (theorist1 @ Jun 13 2013, 23:32) .....   Aug 4 2013, 17:43
- - mzil   QUOTE And what level-roving experiments show is th...   Aug 4 2013, 22:54
|- - Woodinville   QUOTE (mzil @ Aug 4 2013, 14:54) however ...   Aug 5 2013, 03:22
- - greynol   How can something that serves to make distinguishi...   Aug 5 2013, 04:33
|- - mzil   QUOTE (greynol @ Aug 4 2013, 23:33) How c...   Aug 6 2013, 17:15
|- - Kees de Visser   QUOTE (mzil @ Aug 6 2013, 18:15) Kees de ...   Aug 22 2013, 08:59
|- - Woodinville   QUOTE (Kees de Visser @ Aug 22 2013, 00:5...   Aug 24 2013, 13:11
|- - Arnold B. Krueger   QUOTE (Woodinville @ Aug 24 2013, 08:11) ...   Aug 25 2013, 01:21
- - 2Bdecided   Two mono samples, A and B. sequential: Listen to ...   Aug 5 2013, 10:46
- - Wayne Highwood   In light of the limits of serial audio testing rai...   Aug 5 2013, 19:14
|- - Arnold B. Krueger   QUOTE (Wayne Highwood @ Aug 5 2013, 14:14...   Aug 13 2013, 01:38
- - greynol   I see a distinction without a difference. Subject...   Aug 5 2013, 19:23
|- - Wayne Highwood   QUOTE (greynol @ Aug 5 2013, 12:23) I see...   Aug 5 2013, 19:59
- - 2Bdecided   So you are exploring the possibility that the reas...   Aug 5 2013, 20:59
|- - Wayne Highwood   QUOTE (2Bdecided @ Aug 5 2013, 13:59) So ...   Aug 5 2013, 22:05
|- - saratoga   QUOTE (Wayne Highwood @ Aug 5 2013, 17:05...   Aug 5 2013, 22:27
- - 2Bdecided   People who look for reasons to doubt DBTs generall...   Aug 6 2013, 10:15
|- - Wayne Highwood   QUOTE (2Bdecided @ Aug 6 2013, 03:15) Peo...   Aug 6 2013, 16:47
|- - [JAZ]   QUOTE (Wayne Highwood @ Aug 6 2013, 17:47...   Aug 6 2013, 17:45
|- - 2Bdecided   QUOTE (Wayne Highwood @ Aug 6 2013, 16:47...   Aug 6 2013, 18:16
|- - Wayne Highwood   I just hope you guys in the industry don't end...   Aug 6 2013, 21:13
|- - Wayne Highwood   QUOTE (2Bdecided @ Aug 6 2013, 11:16) * -...   Aug 15 2013, 00:31
|- - greynol   QUOTE (Wayne Highwood @ Aug 14 2013, 16:3...   Aug 15 2013, 00:45
|- - Woodinville   QUOTE (Wayne Highwood @ Aug 14 2013, 16:3...   Aug 15 2013, 01:01
||- - Wayne Highwood   QUOTE (Woodinville @ Aug 14 2013, 18:01) ...   Aug 15 2013, 14:39
||- - Woodinville   QUOTE (Wayne Highwood @ Aug 15 2013, 06:3...   Aug 17 2013, 04:57
|- - Arnold B. Krueger   QUOTE (Wayne Highwood @ Aug 14 2013, 19:3...   Aug 22 2013, 13:14
- - greynol   The point is that ABX is useful for more than the ...   Aug 15 2013, 15:08
|- - Wayne Highwood   QUOTE (greynol @ Aug 15 2013, 08:08) The ...   Aug 15 2013, 16:21
|- - saratoga   QUOTE (Wayne Highwood @ Aug 15 2013, 11:2...   Aug 15 2013, 19:26
- - greynol   I live on the West Coast so there's still time...   Aug 17 2013, 01:10
- - Arnold B. Krueger   QUOTE (theorist1 @ Jun 13 2013, 22:32) As...   Nov 28 2014, 17:50
- - ajinfla   I fear the battle is lost on the issue of honesty ...   Dec 9 2014, 23:49


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd December 2014 - 10:24