IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3 >  
Reply to this topicStart new topic
Biophysics, Limitations of Shannon and Issues with ABX Testing, Split from Topic ID: 57406 (TOS #5)
knutinh
post Jun 18 2013, 13:01
Post #26





Group: Members
Posts: 570
Joined: 1-November 06
Member No.: 37047



QUOTE (Porcus @ Jun 18 2013, 10:41) *
Now is the hearing upper limint of human hearing completely independent of waveform, or is it not?

It appears that humanity, so far, have been unable to show audible differences between the CD format and the DVD-A/SACD formant, a prominent difference between the two is that the latter allows extension far beyond the 20kHz or so of CD.

One would think that the variation of waveshape found in such studies would be sufficient, although the amount of energy beyond 20kHz tends to be low (as is the case with most live music).

-k
Go to the top of the page
+Quote Post
Porcus
post Jun 18 2013, 23:26
Post #27





Group: Members
Posts: 1993
Joined: 30-November 06
Member No.: 38207



QUOTE (knutinh @ Jun 18 2013, 14:01) *
It appears that humanity, so far, have been unable to show audible differences between the CD format and the DVD-A/SACD formant, a prominent difference between the two is that the latter allows extension far beyond the 20kHz or so of CD.


.... indicating that the CD format is “more than good enough”, but not what would be “precisely good enough”.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
knutinh
post Jun 19 2013, 06:24
Post #28





Group: Members
Posts: 570
Joined: 1-November 06
Member No.: 37047



QUOTE (Porcus @ Jun 18 2013, 23:26) *
QUOTE (knutinh @ Jun 18 2013, 14:01) *
It appears that humanity, so far, have been unable to show audible differences between the CD format and the DVD-A/SACD formant, a prominent difference between the two is that the latter allows extension far beyond the 20kHz or so of CD.


.... indicating that the CD format is “more than good enough”, but not what would be “precisely good enough”.

Indicating that the decades of research into human auditory perception is right, a lowpass filter at 15 kHz or 25 kHz or whatever (depending on age, gender, genes, ...) seems to be transparent. Any claims that humans can somehow take advantage of ultra-sound in certain non-linear, waveform-dependent ways are less believable as a result.

At my age, 15 kHz tends to be sufficient.

-k
Go to the top of the page
+Quote Post
Woodinville
post Jun 20 2013, 06:26
Post #29





Group: Members
Posts: 1414
Joined: 9-January 05
From: In the kitchen
Member No.: 18957



QUOTE (Porcus @ Jun 18 2013, 02:41) *
Now is the hearing upper limint of human hearing completely independent of waveform, or is it not?


Uh, of course it's not "independent of waveform" since the domain the ear responds in is 4th order resonance, give or take, and that is going to map directly to a fourier basis.

So you need to use a fourier basis, not "any waveform".


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
krabapple
post Jun 20 2013, 11:17
Post #30





Group: Members
Posts: 2514
Joined: 18-December 03
Member No.: 10538



QUOTE (theorist1 @ Jun 14 2013, 22:11) *
ly. In principle, one could play a pristine recording, and that same recording to which very low levels of a very irritating form of audio distortion had been added, while monitoring a specific physiological attribute. With a sufficiently large N, and with the right attribute, one might find significant differences in physiological response to the pristine and non-pristine versions, even if the subjects subsequently failed to distinguish them based on ABX testing.


And away we go.... rolleyes.gif

http://en.wikipedia.org/wiki/Hypersonic_effect
Go to the top of the page
+Quote Post
antz
post Aug 4 2013, 13:46
Post #31





Group: Members
Posts: 240
Joined: 26-March 05
From: Wolverhampton UK
Member No.: 20961



QUOTE (theorist1 @ Jun 14 2013, 04:32) *
If it is found that subjects indeed fail to reliably distinguish colors when presented sequentially that they can reliably distinguish when presented simultaneously, this will suggest that sequential ABX testing may not be a good method for assessing our perceptual limits (i.e., for determining transparency) -- and not just for vision, but possibly for hearing as well.

I'm a little late to this discussion but the partial quote above seems to contain a fundamental flaw, my highlighting.

I would suggest that "sequential" ABXing and "simultaneous" ABXing of colours would show a large discrepancy because most people have very poor colour memory. I spent a large portion of my working life in colour-related industry and only ever came across a single person whose colour-memory was uncannily good. That aside, it surely is a flawed idea that hearing must be also be so afflicted, unless there is evidence to suggest it. Is there any evidence either way?

In addition, it is patently obvious that "simultaneous" ABXing is possible with coloured tiles, whereas with audio samples it is inherently somewhere between difficult and impossible. The fact that a person's ear could discriminate between samples if they were simultaneous is irrelevant since that isn't feasible. What matters is if they can "hold" the audio memory or not.

Let me suggest a more appropriate analogy to colour. Let's say you decide to re-paint your living room walls. You pay a visit to a paint supplier and select your desired colour from a chart. A machine then dispenses the required mixture and off you go and paint your walls. Knowing the technology involved, I can almost guarantee the walls will not match the chart exactly. Will you notice this? Highly unlikely, assuming there was no grave error in the process. Would you be able to ABX the chart and your walls? Highly likely. Does this make the wall-painting process "non-transparent", in the audio-ABX sense? Not a bit.
Go to the top of the page
+Quote Post
Light-Fire
post Aug 4 2013, 17:43
Post #32





Group: Members
Posts: 424
Joined: 5-August 06
From: Canada
Member No.: 33645



QUOTE (theorist1 @ Jun 13 2013, 23:32) *
...sequential ABX testing is hard, because it requires accurate memory of both A and B when judging X...


A NON blind test also requires accurate memory of both A and B. So your argument is irrelevant.
Go to the top of the page
+Quote Post
greynol
post Aug 4 2013, 18:31
Post #33





Group: Super Moderator
Posts: 10335
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (theorist1 @ Jun 14 2013, 04:32) *
If

http://www.hydrogenaudio.org/forums/index....st&p=838137


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Woodinville
post Aug 4 2013, 21:42
Post #34





Group: Members
Posts: 1414
Joined: 9-January 05
From: In the kitchen
Member No.: 18957



QUOTE (antz @ Aug 4 2013, 05:46) *
The fact that a person's ear could discriminate between samples if they were simultaneous is irrelevant since that isn't feasible.


And what level-roving experiments show is that you want clean, clickless near-instantaneous switching, no more, no less.

As to the title, the Shannon theorem is right, and ABX testing is as sensitive as anything else when it's done right.

I've seen many miserable excuses for why that isn't, but I am frankly tired of them, it's the same old mistakes over and over again.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
antz
post Aug 4 2013, 22:14
Post #35





Group: Members
Posts: 240
Joined: 26-March 05
From: Wolverhampton UK
Member No.: 20961



QUOTE (Woodinville @ Aug 4 2013, 21:42) *
QUOTE (antz @ Aug 4 2013, 05:46) *
The fact that a person's ear could discriminate between samples if they were simultaneous is irrelevant since that isn't feasible.


And what level-roving experiments show is that you want clean, clickless near-instantaneous switching, no more, no less.

As to the title, the Shannon theorem is right, and ABX testing is as sensitive as anything else when it's done right.

I've seen many miserable excuses for why that isn't, but I am frankly tired of them, it's the same old mistakes over and over again.

I didn't word that well - a better wording would have been: The fact that a person's ear might be able to discriminate between samples if they were simultaneous is irrelevant since that isn't feasible.
Go to the top of the page
+Quote Post
mzil
post Aug 4 2013, 22:54
Post #36





Group: Members
Posts: 729
Joined: 5-August 07
Member No.: 45913



QUOTE
And what level-roving experiments show is that you want clean, clickless near-instantaneous switching, no more, no less

In my experience, based on my perception, lightning fast transitions are indeed critically important to being able to audibly discern subtle differences between samples [such as level or minor tonal balance variations] however I've always assumed that the reason we want to avoid transitional clicks between A, B, and X is because they potentially may act as a "tell", a giveaway as to the samples' identity, not because hearing the minor clicks themselves diminishes our sensitivity to picking up on small differences. Correct?

What I'm getting at is that consistent, minor ticks/clicks which are first verified, pre-test, to be audibly identical each time one switches between samples [so it can't act as a "tell"] is perhaps not ideal, however it is far less destructive to our sensitivity, in my mind, than another solution I've sometimes seen used to overcome minor clicks in some tests, where a quick volume ramp is introduced at each transition to obscure any such clicks. The problem being the time delay introduced by that volume up ramp, even if it is only one second in duration, reduces our sensitivity to hearing minor changes, significantly. [Also, hearing the volume ramp itself, or a beep, or pink noise, or any alternative sounds to the targets, for that matter, may simply throw off our concentration and focus to the task at hand.]

This post has been edited by mzil: Aug 4 2013, 23:31
Go to the top of the page
+Quote Post
Woodinville
post Aug 5 2013, 03:22
Post #37





Group: Members
Posts: 1414
Joined: 9-January 05
From: In the kitchen
Member No.: 18957



QUOTE (mzil @ Aug 4 2013, 14:54) *
however I've always assumed that the reason we want to avoid transitional clicks between A, B, and X is because they potentially may act as a "tell", a giveaway as to the samples' identity, not because hearing the minor clicks themselves diminishes our sensitivity to picking up on small differences. Correct?


No, actually, if you'll think briefly about the loudness of a click, in addition to being a tell, it ALSO mucks up short-term loudness memory, so it reduces sensitivity AND provides a leak in the DBT protocol. It's a twofer.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
greynol
post Aug 5 2013, 04:33
Post #38





Group: Super Moderator
Posts: 10335
Joined: 1-April 04
From: San Francisco
Member No.: 13167



How can something that serves to make distinguishing two samples more difficult also serve as a tell?

It doesn't.

ABX testing, as I know it, hands the power of switching to the testee. A click will only be a tell if it occurs for one sample and not the other for both the unknown X and the corresponding known (A or B, but not both). That a click occurs as the result of switching is meaningless in alerting the testee to the switch since the testee initiated it.

This post has been edited by greynol: Aug 5 2013, 04:55


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
2Bdecided
post Aug 5 2013, 10:46
Post #39


ReplayGain developer


Group: Developer
Posts: 5353
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Two mono samples, A and B.

sequential: Listen to A then B.

simultaneous: Listen to A in one ear and B in the other.

Some differences will be far easier to hear in the simultaneous presentation merely due to the binaural masking level difference (BMLD).

I can't think of another way of doing simultaneous presentation that isn't a nonsense, and that one is slightly a nonsense because mostly what you're measuring is BMLD, not simultaneous vs sequential.

Cheers,
David.
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 5 2013, 19:14
Post #40





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



In light of the limits of serial audio testing raised by the OP, I'm pondering two possible conclusions:

"We've reached the limits of differences the subjects can hear"

vs.

"We have reached the limits of resolution for using human memory as the measuring instrument in an AB test".

Though the OP's point is generating some discussion, it seems to be sidestepping his basic criticism (which seems logical, particularly when considering current knowledge about perception).

Thoughts?
Go to the top of the page
+Quote Post
greynol
post Aug 5 2013, 19:23
Post #41





Group: Super Moderator
Posts: 10335
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I see a distinction without a difference. Subjective interpretation requires memory. If memory is suspect then so are the impressions born from it.

This was already addressed, was it not?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 5 2013, 19:59
Post #42





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



QUOTE (greynol @ Aug 5 2013, 12:23) *
I see a distinction without a difference. Subjective interpretation requires memory. If memory is suspect then so are the impressions born from it.

This was already addressed, was it not?


I guess not in the way I was hoping or expecting from the dbt experts. (Great forum, by the way. I'm not trolling, just trying to learn and generate some discussion. Hope this horse ain't dead.)

DBT fails to address the "unreliable memory" problem in any way, as it relies on such for it's data input. The tests indicate that gross differences are clearly audible. As those differences shrink, more and more fail the test. When you reach the useful resolution of a memory based AB test, most people fail. But then which conclusion posted above should be drawn from the results? It seems an important distinction, as the validity of the method is assumed in the first while the second one explicitly acknowledges it's (possible, likely?) limits.







This post has been edited by Wayne Highwood: Aug 5 2013, 20:40
Go to the top of the page
+Quote Post
2Bdecided
post Aug 5 2013, 20:59
Post #43


ReplayGain developer


Group: Developer
Posts: 5353
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



So you are exploring the possibility that the reason no difference is reported is not because we cannot hear the difference, but because we cannot remember it for a few seconds.

If we have forgotten it before we have even drawn another breath, how do we subsequently know that we ever heard it?

Cheers,
David.
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 5 2013, 22:05
Post #44





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



QUOTE (2Bdecided @ Aug 5 2013, 13:59) *
So you are exploring the possibility that the reason no difference is reported is not because we cannot hear the difference, but because we cannot remember it for a few seconds.


Second clause mainly. I don't want you or anyone to read my posts as any sort of apologia to the audiophile crowd. I'm more curious if the tools being used to support claims are properly calibrated, so to speak. The serial listening criticism seems to have teeth, even if it's off in the academic weeds for all practical purposes.

QUOTE (2Bdecided @ Aug 5 2013, 13:59) *
If we have forgotten it before we have even drawn another breath, how do we subsequently know that we ever heard it?


Now you're getting a bit too far into the weeds. I did not want an epistemic discussion of what we know, I simply wanted to explore what seems like a legit criticism of dbt from those in the know.

This post has been edited by Wayne Highwood: Aug 5 2013, 22:13
Go to the top of the page
+Quote Post
saratoga
post Aug 5 2013, 22:27
Post #45





Group: Members
Posts: 5147
Joined: 2-September 02
Member No.: 3264



QUOTE (Wayne Highwood @ Aug 5 2013, 17:05) *
Now you're getting a bit too far into the weeds. I did not want an epistemic discussion of what we know, I simply wanted to explore what seems like a legit criticism of dbt from those in the know.


Generally when ABXing, you can switch rapidly (even instantly) between samples, so if you're going to make this argument you would be diving into the weeds and arguing that an instant is too long.

I think mainly this is just an argument against setting up tests poorly such that subjects cannot easily switch between samples. With proper setup, this should not be an issue.
Go to the top of the page
+Quote Post
2Bdecided
post Aug 6 2013, 10:15
Post #46


ReplayGain developer


Group: Developer
Posts: 5353
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



People who look for reasons to doubt DBTs generally believe they can hear an audible difference sighted but cannot hear an audible difference blind. They are trying to discover some difference in the testing methodology (other than sighted-vs-blind) to justify their belief that they really hear a real audible difference. They want this other difference in testing methodology to be the reason the believed audible difference vanishes in blind testing, rather than accepting that it vanishes simply due to the blinding.

In both kinds of tests, people need to rely on their memory. If there is any difference, it is that they need to rely on memory less so in some DBT designs (as saratoga says), not more. Hence reliance on memory is not this "other" testing methodology difference that they seek.

In short, I think you're barking up the wrong tree.

Cheers,
David.
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 6 2013, 16:47
Post #47





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



QUOTE (2Bdecided @ Aug 6 2013, 03:15) *
People who look for reasons to doubt DBTs generally believe they can hear an audible difference sighted but cannot hear an audible difference blind. They are trying to discover some difference in the testing methodology (other than sighted-vs-blind) to justify their belief that they really hear a real audible difference. They want this other difference in testing methodology to be the reason the believed audible difference vanishes in blind testing, rather than accepting that it vanishes simply due to the blinding.

In both kinds of tests, people need to rely on their memory. If there is any difference, it is that they need to rely on memory less so in some DBT designs (as saratoga says), not more. Hence reliance on memory is not this "other" testing methodology difference that they seek.

In short, I think you're barking up the wrong tree.

Cheers,
David.


I'm not one who claims to hear differences, nor do I really care much about those that do. What I'm trying to discuss is not people suffering delusions, but the validity and reliability of a test based on human sensory impressions/conscious awareness/memory as the source of data.

I guess I like my data to be more raw and empirical. Brains are really amazing biological devices, but they make for piss poor lab equipment.
Go to the top of the page
+Quote Post
mzil
post Aug 6 2013, 17:15
Post #48





Group: Members
Posts: 729
Joined: 5-August 07
Member No.: 45913



QUOTE (greynol @ Aug 4 2013, 23:33) *
How can something that serves to make distinguishing two samples more difficult also serve as a tell?

When the click noise has not been verified, pre-test, to be uniform in its sound quality (and/or timing) when transitioning to either A or B; then it can act as a tell.

Arnold B. Krueger mentions this problem can exist in hardware ABX testing and personally being able to reliably distinguish the sound of the QSC ABX switcher transitions, at least in the absence of music playing, here. He also mentions it occasionally can be an issue with PCABX (at least when using certain computers) in that post.

A slightly different but equally problematic issue is when the click sound doesn't immediately identify the signal itself, however is does indicate to the listener if the transition was to the same signal vs. the alternate signal [For example, the artifact click sound switching from A to X, where X = A, vs. the sound of A to X where instead X = B].

Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.

The unrelated reason why we ideally don't want clicks at all, is because the presence of any other obtrusive sound, during the transition period, be it a blast from a bull horn, an annoying loud beep, or even just a loud click/thump can muck up our short term memory for things like loudness (or I would assume, also tonality). This I would assume is rather self evident. Sorry if I wasn't more clear about the "tell" part.
Go to the top of the page
+Quote Post
Martel
post Aug 6 2013, 17:20
Post #49





Group: Members
Posts: 559
Joined: 31-May 04
From: Czech Rep.
Member No.: 14430



QUOTE (2Bdecided @ Jun 14 2013, 10:38) *
Except comparing two mono signals simultaneously, one fed to each ear via headphones. A niche pass-time, but possible. wink.gif
This generally won't work because human body is rarely perfectly symmetrical (similar for headphones). Left and right ear may deteriorate (be damaged) at differing rates etc.
For example, when I activate the channel swap plugin in Foobar2000, it's not just side-swap for me. It's likely because I recently damaged my right ear a bit in a gym (we were dropping the weight carelessly while dead-lifting).


--------------------
IE4 Rockbox Clip+ AAC@192; HD 668B/HD 518 Xonar DX FB2k FLAC;
Go to the top of the page
+Quote Post
[JAZ]
post Aug 6 2013, 17:45
Post #50





Group: Members
Posts: 1797
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



QUOTE (Wayne Highwood @ Aug 6 2013, 17:47) *
I guess I like my data to be more raw and empirical. Brains are really amazing biological devices, but they make for piss poor lab equipment.


We do not want raw data or data that simply demonstrates that the subject has received a stimulus from the outside world. There are things that we perceive, but that we aren't conscious about that perception (for example, when paying attention to something complex, we might not be conscious of small changes around us), and there are actions we do which we don't control them directly (like heart beating or breathing, although we can partially stop the second one).

As such, we rely on what the subject is able to tell that he has perceived. Since the question is made just after the perception has been received, the idea of forgeting it seems absurd (in a sane individual), and, at much, we can wonder how accurate the communication of what he has perceived is. That is not much different than calibrating a device. (except there are more variables).

This post has been edited by [JAZ]: Aug 6 2013, 17:46
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th December 2014 - 21:12