IPB

Welcome Guest ( Log In | Register )

DBT Is Flawed Because Bob Stuart Says So, Split from Topic ID #11442
2Bdecided
post Aug 6 2012, 11:22
Post #1


ReplayGain developer


Group: Developer
Posts: 5147
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I don't think it's helpful criticising someone for not running proper DBTs of loudspeakers. As far as I know, only one or two members here have tried.

I know the loudspeaker comments were part of a longer post that had many other issues, but I don't think ABX-ing loudspeakers is a realistic thing to jump on.

Cheers,
David.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
audioclaudio
post Aug 8 2012, 20:17
Post #2





Group: Members
Posts: 33
Joined: 3-December 11
Member No.: 95560



The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part. This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.
Often, if not practically always, it is too time consuming and/or too uneconomical to conduct an ABX test properly, i.e. in such way that expectation bias doesn't creep in through the back door etcetera. Experts in auditory neuroscience and psychoacoustics have gathered experimental evidence which appears to indicate the following. Humans who remember different things are perfectly capable of hearing the same sounds differently as a result of remembering different things. For example, Bob Stuart of Meridian, who has a Ph.D in neuroscience, believes that it is perfectly possible for a person to not hear a specific detail in a piece of music when it is played back on one particular system "A", then to discover this specific detail by listening to the same music again on a better, more resolving system "B" next, and then, finally, to turn back to the previous system "A" and always hear this detail on system "A" even though the detail could previously not be heard on system "A". Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.
Go to the top of the page
+Quote Post
[JAZ]
post Aug 11 2012, 16:20
Post #3





Group: Members
Posts: 1783
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



audioclaudio and Arnold : These two last pages of the thread are really difficult to follow, and don't really help at all to make a decision that answers the topic's title.


First of all, I would like to rewind and set a straight and concrete meaning of several words, since it's not really clear what you're against:

- Objective test/Objectivism ( http://en.wikipedia.org/wiki/Objectivism . In this thread, really http://en.wikipedia.org/wiki/Objectivity_%28science%29 ) : The main idea behind this is that the person doing the test does not matter, since the reality exists independently of the subject taking the test (1 meter is 1 meter no matter who checks it) . Objectivity does make possible to repeat tests and *verify* a result.

- Subjective test/Subjectivism ( http://en.wikipedia.org/wiki/Subjectivism ) : The main idea under subjectivism is that we know the world thanks to our sensory system, and as such, only what we can perceive is what exists (ok. This is just one interpretation. Subjectivism has many variants, from phylosophical, to theological).


- ABX Test: An Objective test that evaluates the response (subjective or not) of the participants to the items at test, and obtains a (statistically) representable result that can be considered reliable and, as such, valid for the specific group at which the study was directed.

- ABX Test, application: An ABX test can only say that a difference exists between the two items being evaluated by the determined subgroup at which the study is directed. (Concretely, this means that it can never be used to say that a difference does not exist. At much, with many failed tests, it can be an indication that such difference is not probable).

In case of Hydrogenaudio, this means that it allows us to *verify* that an *audible difference* between two audio samples (being them two different codecs, two soundcards, two amplifiers....) does exist for the person or people involved in doing the ABX test. If enough tests validate this, then, and only then, can be considered that the difference is real and not due to some error in the test.


Hydrogenaudio is an Objectivism Forum. Let's put that straight and undeniably clear. The high emphasis on ABX tests, the relation with the science world, and the participation of people directly related to the subject of this forum (like the actual coders of audio codecs) is an indication of it.



Reasons why ABX tests are a requirement in order to get objective tests, reliable and representable for the intended audience:


A) Expectation bias ( http://en.wikipedia.org/wiki/Expectation_bias ): A subject, intentionally or not, can be influenced by its subconsciusness: Someone listening to a song in a newly bought equipment does have expectation bias when evaluating its sound quality against an older, less good equipment. (while a difference might exists, the subject could be perceiving his feeling that a difference exists and not the real difference).

B) Representative: To pass an ABX test, it is required to correctly identify a percentage of the trials, and they cannot be cherry picked. ABX pretends to reduce the possibilities of: chance/luck, hidden elements at test (i.e. a difference caused by something that is not the item at test), unconscious influence from other subjects (that's the requirement for the test to be *double* blind).


Setting up an audio ABX test is not necesarily easy, Here are some examples of what can go wrong:

- Playing at different sample rates: soundcards, soundcard drivers, OS audio stack, or even the playback application can cause problems not related to the original audio, but to the different sampling rate being used.
- Volume mismatch: It is important that the samples at test are correctly leveled, so that the subject evaluates the audio, not the difference in audio amplitude. Differences of less than 1dB can be audible.
- Clipping/distortions caused by playback settings/equipment: It is not strange that audio clips digitally today. While the clip in itself could be heard and could determine a limitation or problem depending on the intended use, the problem could be gone just by setting the amplification properly.



When is an ABX necessary:

ABX test, as said above, help to determine that a difference exists. For Hydrogenaudio it is required when an *audible* difference is reported and:

- It is not commonly known that it exists
- It is against the general knowledge that says such difference should not be audible.
- It is not measurable by other means that could clearly determine otherwise (i.e. an audible difference in lossless codecs can be discarded if the codec works and so the problem, if any, exists somewhere else).



To finish this post, I'd like to add that ABX is not intended to avaluate if an item is better than another. In order to determine such thing, it is necessary that they are found to be different. Once that is verified, the difference can be avaluated between both items.

It's been said that ABX does not impose a limitation of how long/short a listening period runs for, neither puts it limits in how many times you switch back an forth to determine a difference/preference.
Also note that the existence of a difference is objective. The avaluation of the difference, while some rules apply, is more subjective.
That's why on the public listening tests, the statistic methods (like MUSHRA) that are applied on the final results of the group try to reduce the influence of subjective decisions.

This post has been edited by [JAZ]: Aug 11 2012, 16:32
Go to the top of the page
+Quote Post

Posts in this topic
2 Pages V   1 2 >


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 30th September 2014 - 18:27