IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3  
Reply to this topicStart new topic
Biophysics, Limitations of Shannon and Issues with ABX Testing, Split from Topic ID: 57406 (TOS #5)
2Bdecided
post Aug 6 2013, 18:16
Post #51


ReplayGain developer


Group: Developer
Posts: 5176
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (Wayne Highwood @ Aug 6 2013, 16:47) *
What I'm trying to discuss is not people suffering delusions, but the validity and reliability of a test based on human sensory impressions/conscious awareness/memory as the source of data.
...but we're testing whether people using their sensory impressions and conscious awareness* can remember hearing a difference.

That's relevant to the choice of audio codec or hi-fi equipment.

* - I included "conscious awareness" just to copy your sentence, but I don't think it's necessarily a restriction of DBTs. Many people who do ABX testing say it reveals audible differences that they were barely if at all "consciously" aware of. They thought they were guessing throughout, but they could guess correctly with statistical significance.

QUOTE
I guess I like my data to be more raw and empirical. Brains are really amazing biological devices, but they make for piss poor lab equipment.
...but we listen to audio codecs or hi-fi equipment with our ears and brains, not lab equipment.

I think what you are wanting is irrelevant to the task at hand. The point of hi-fi, audio coding, Hydrogenaudio etc is to listen to audio (music, speech, whatever).

The question you are asking seems to be quite different. It might make an interesting scientific study, but I don't think you've made a convincing argument that it has any relevance to listening to music.

Cheers,
David.

This post has been edited by 2Bdecided: Aug 6 2013, 18:16
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 6 2013, 21:13
Post #52





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



I just hope you guys in the industry don't end up with the wrong codec because you're using a rubber yardstick. (I kid.) I have it easy. I've been moved to tears by a highly compressed Beethoven's 9th on a table radio.

Thanks for the interesting discussion, and for not banning me for random questions from the peanut gallery!

This post has been edited by Wayne Highwood: Aug 6 2013, 21:16
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Aug 13 2013, 01:38
Post #53





Group: Members
Posts: 3962
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (Wayne Highwood @ Aug 5 2013, 14:14) *
In light of the limits of serial audio testing raised by the OP, I'm pondering two possible conclusions:

"We've reached the limits of differences the subjects can hear"

vs.

"We have reached the limits of resolution for using human memory as the measuring instrument in an AB test".

Though the OP's point is generating some discussion, it seems to be sidestepping his basic criticism (which seems logical, particularly when considering current knowledge about perception).



Do you have some means for comparing two sounds that does not depend on human memory? Please do tell!
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 15 2013, 00:31
Post #54





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



QUOTE (2Bdecided @ Aug 6 2013, 11:16) *
* - I included "conscious awareness" just to copy your sentence, but I don't think it's necessarily a restriction of DBTs. Many people who do ABX testing say it reveals audible differences that they were barely if at all "consciously" aware of. They thought they were guessing throughout, but they could guess correctly with statistical significance.


That is an interesting observation. Lots more sensory data is received and processed than what actually get's through to conscious awareness. I wonder if the subjects could maintain their batting average over a larger sample size.

QUOTE (Arnold B. Krueger @ Aug 12 2013, 18:38) *
Do you have some means for comparing two sounds that does not depend on human memory? Please do tell!


Can't really think of one. But for that matter I am not trying to develop audibly transparent compression codecs.

What I wonder is this: can it be done simply by the numbers? We know thresholds of audibility, why not design to that, instead of the less precise ability of humans to differentiate? Is it because of actual loss of data in the compression schemes, a matter of finding that happy place between the fat milk and the skim that folks will still find palatable?

Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 00:45
Post #55





Group: Super Moderator
Posts: 10043
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Wayne Highwood @ Aug 14 2013, 16:31) *
We know thresholds of audibility, why not design to that

What makes you think this isn't done?

Something is used to determine where to reduce precision in order to make a signal easier to compress, right?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Woodinville
post Aug 15 2013, 01:01
Post #56





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Wayne Highwood @ Aug 14 2013, 16:31) *
QUOTE (Arnold B. Krueger @ Aug 12 2013, 18:38) *
Do you have some means for comparing two sounds that does not depend on human memory? Please do tell!


Can't really think of one. But for that matter I am not trying to develop audibly transparent compression codecs.


What ever does what Arnold said have to do with what you said?


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 15 2013, 14:39
Post #57





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



QUOTE (Woodinville @ Aug 14 2013, 18:01) *
What ever does what Arnold said have to do with what you said?


We're discussing serial audio testing of human subjects, which necessarily relies on memory.

Arny asked if I had means of comparing sound without using memory, I replied in the negative (assuming human subjects again, not the use of some sort of measurement equipment). I also pointed out that I am not developing anything that requires discrimination testing of this sort.











Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 15:08
Post #58





Group: Super Moderator
Posts: 10043
Joined: 1-April 04
From: San Francisco
Member No.: 13167



The point is that ABX is useful for more than the development of lossy codecs.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Wayne Highwood
post Aug 15 2013, 16:21
Post #59





Group: Members
Posts: 12
Joined: 5-August 13
Member No.: 109478



QUOTE (greynol @ Aug 15 2013, 08:08) *
The point is that ABX is useful for more than the development of lossy codecs.


I get that.

The rather confrontational, terse one-line interrogatives and comments gives the distinct impression you guys are preparing to tee off on me, which is great if that sort of behavior meets your psychosocial needs, but it would be like a college professor demeaning a freshman. I just visited to learn, not be treated with scorn. Have a nice day, fellas.
Go to the top of the page
+Quote Post
saratoga
post Aug 15 2013, 19:26
Post #60





Group: Members
Posts: 5047
Joined: 2-September 02
Member No.: 3264



QUOTE (Wayne Highwood @ Aug 15 2013, 11:21) *
QUOTE (greynol @ Aug 15 2013, 08:08) *
The point is that ABX is useful for more than the development of lossy codecs.


I get that.


Then your reply above doesn't really make sense, or perhaps you have misunderstood what you've quoted above.

QUOTE (Wayne Highwood @ Aug 15 2013, 11:21) *
The rather confrontational, terse one-line interrogatives and comments gives the distinct impression you guys are preparing to tee off on me,


Hello and welcome to the internet where people are not always going to agree with you.
Go to the top of the page
+Quote Post
greynol
post Aug 17 2013, 01:10
Post #61





Group: Super Moderator
Posts: 10043
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I live on the West Coast so there's still time, and I definitely will. Thanks.

My apologies for not noticing that you stopped carrying the torch for this discussion as of 8/6.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Woodinville
post Aug 17 2013, 04:57
Post #62





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Wayne Highwood @ Aug 15 2013, 06:39) *
QUOTE (Woodinville @ Aug 14 2013, 18:01) *
What ever does what Arnold said have to do with what you said?


We're discussing serial audio testing of human subjects, which necessarily relies on memory.

Arny asked if I had means of comparing sound without using memory, I replied in the negative (assuming human subjects again, not the use of some sort of measurement equipment). I also pointed out that I am not developing anything that requires discrimination testing of this sort.


If you want to do an audio test, how else would you do this? By the way, sequential tests with proper windowing are documented as the best way to extract the most reliable answers from subjects.

If you're not interested in audio testing, why are we having this discussion? Seriously. I am confused.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Kees de Visser
post Aug 22 2013, 08:59
Post #63





Group: Members
Posts: 685
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (mzil @ Aug 6 2013, 18:15) *
Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.
Sorry for being late. The artifacts I found are indeed random. They also appear when switching between identical audio, which is strange since this is a trivial task and should be 100% lossless, e.g. with a simple linear fade. The artifacts I was/am worried about will only appear when switching between different audio streams, which is not a lossless process, not a trivial task and might be audible, depending on many variables.
Hope this helps.
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Aug 22 2013, 13:14
Post #64





Group: Members
Posts: 3962
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (Wayne Highwood @ Aug 14 2013, 19:31) *
What I wonder is this: can it be done simply by the numbers? We know thresholds of audibility, why not design to that, instead of the less precise ability of humans to differentiate? Is it because of actual loss of data in the compression schemes, a matter of finding that happy place between the fat milk and the skim that folks will still find palatable?


In the early 1980s I sat through an AES presentation about the development of lossy encoders based on the thresholds of hearing. It was tough going for the developers. Couldn't get worthwhile amounts of data compression. Later on masking became better understood, and development of lossy encoders shifted into high gear.

The point being that the thresholds of audibility overestimated the working sensitivity of the human ear.

While the golden ears rant and rave about how wrong Fletcher and Munson were, they were actually overly optimistic.


Go to the top of the page
+Quote Post
Woodinville
post Aug 24 2013, 13:11
Post #65





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Kees de Visser @ Aug 22 2013, 00:59) *
QUOTE (mzil @ Aug 6 2013, 18:15) *
Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.
Sorry for being late. The artifacts I found are indeed random. They also appear when switching between identical audio, which is strange since this is a trivial task and should be 100% lossless, e.g. with a simple linear fade. The artifacts I was/am worried about will only appear when switching between different audio streams, which is not a lossless process, not a trivial task and might be audible, depending on many variables.
Hope this helps.


Two things come to mind:

1) bad time alignment, although with 2 identical files that would seem unlikely
2) bad crossfade window design

If the click when switching between A/X is more noticible than B/X, you can be sure subjects will latch on to that.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Aug 25 2013, 01:21
Post #66





Group: Members
Posts: 3962
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (Woodinville @ Aug 24 2013, 08:11) *
QUOTE (Kees de Visser @ Aug 22 2013, 00:59) *
QUOTE (mzil @ Aug 6 2013, 18:15) *
Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.
Sorry for being late. The artifacts I found are indeed random. They also appear when switching between identical audio, which is strange since this is a trivial task and should be 100% lossless, e.g. with a simple linear fade. The artifacts I was/am worried about will only appear when switching between different audio streams, which is not a lossless process, not a trivial task and might be audible, depending on many variables.
Hope this helps.


Two things come to mind:

1) bad time alignment, although with 2 identical files that would seem unlikely
2) bad crossfade window design

If the click when switching between A/X is more noticeable than B/X, you can be sure subjects will latch on to that.


That's for sure! One good test is to run an ABX test with no music playing. If there are switching artifacts or background noises that relate to one alternative but not the other. a careful listener can detect them and do pretty well!

While none of the switchboxes that were sold by the ABX company had this problem, I did encounter a sample of a competitive product that I could score 16/16 with, with nothing attached to it at all.

This sort of problem can be inherent in the switchbox, or it can be due to an error in the setup of the test.

It is also problem to have noises that are truly random. They are a less severe problem but if noticeable enough they can distract the listener and reduce the probability of reliable detection when it is possible.
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th October 2014 - 10:53