IPB

Welcome Guest ( Log In | Register )

4 Pages V  < 1 2 3 4 >  
Reply to this topicStart new topic
What we measure is what we hear
AndyH-ha
post Dec 31 2012, 07:38
Post #26





Group: Members
Posts: 2204
Joined: 31-August 05
Member No.: 24222



Subjective impressions are, after all, impressions relative to the subject, most generally the particular subject only, not relative to the data or the general population.

They might tell us something about the subject, but not much about the object. In some case, subjects even hear what they expect to hear when there is no signal (data/sound) at all. That alone implies the possibility that what they report hearing in the vague terms used in such reports do not relate to anything at all coming from the traducer (speaker/headphone/etc.)
Go to the top of the page
+Quote Post
Woodinville
post Dec 31 2012, 09:43
Post #27





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (AndyH-ha @ Dec 30 2012, 22:38) *
Subjective impressions are, after all, impressions relative to the subject, most generally the particular subject only, not relative to the data or the general population.

They might tell us something about the subject, but not much about the object. In some case, subjects even hear what they expect to hear when there is no signal (data/sound) at all. That alone implies the possibility that what they report hearing in the vague terms used in such reports do not relate to anything at all coming from the traducer (speaker/headphone/etc.)


A properly designed and operated subjective test IS a form of measurement.

Audiophile tests generally (almost always) need not apply, of course.

But, a subjective test, run as a DBT, using a proper design, with anchors and controls, is a measurement. A measurement that is a true pain in the behind to do, perhaps, but it IS a measurement, and it most often even gives a useful noise level for the measurement.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
greynol
post Dec 31 2012, 11:59
Post #28





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



TL;DR:
MUSHRA and other ABC-style tests are subjective tests. This is because the person taking the test assigns a score to each stimuli in order to declare a personal preference.


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
Satellite_6
post Dec 31 2012, 15:32
Post #29





Group: Members
Posts: 37
Joined: 13-September 10
From: VA, USA
Member No.: 83831



So far the best measuring headphones I have tried sound the best and the best measuring amps sound the best so there is no contradiction between what sounds good and what measures well to me. It makes it easy.


--------------------
People are silly.
Go to the top of the page
+Quote Post
Martel
post Dec 31 2012, 16:15
Post #30





Group: Members
Posts: 553
Joined: 31-May 04
From: Czech Rep.
Member No.: 14430



What's the "best measuring" frequency response of a headphone? I wonder... smile.gif


--------------------
IE4 Rockbox Clip+ AAC@192; HD 668B/HD 518 Xonar DX FB2k FLAC;
Go to the top of the page
+Quote Post
Nessuno
post Dec 31 2012, 16:18
Post #31





Group: Members
Posts: 422
Joined: 16-December 10
From: Palermo
Member No.: 86562



QUOTE (Satellite_6 @ Dec 31 2012, 15:32) *
So far the best measuring headphones I have tried sound the best and the best measuring amps sound the best so there is no contradiction between what sounds good and what measures well to me. It makes it easy.

AKA "expectation bias"... it rarely misses a shot!

P.S. What you said is not wrong in principle, but fact is that human ear is not linear neither has infinite precision... wink.gif

This post has been edited by Nessuno: Dec 31 2012, 16:20


--------------------
... I live by long distance.
Go to the top of the page
+Quote Post
Soap
post Dec 31 2012, 16:42
Post #32





Group: Members
Posts: 1005
Joined: 19-November 06
Member No.: 37767



QUOTE (Martel @ Dec 31 2012, 10:15) *
What's the "best measuring" frequency response of a headphone? I wonder... smile.gif


The one which best matches your ear shape, of course!

wink.gif


--------------------
Creature of habit.
Go to the top of the page
+Quote Post
Kees de Visser
post Dec 31 2012, 17:08
Post #33





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (Woodinville @ Dec 31 2012, 09:43) *
A properly designed and operated subjective test IS a form of measurement.
I thought the food industry is doing this all the time:
http://www.ift.org/
QUOTE
Sensory evaluation has been defined as "a scientific discipline used to evoke, measure, analyze, and interpret reactions to those characteristics of foods and materials as they are perceived by the senses of sight, smell, taste, touch, and hearing" (IFT, 1975).

Perhaps we should be less scared to be subjective. All humans "suffer" from it. The real art is in finding out which variables are responsible.

Happy new (multisensory) year !
Go to the top of the page
+Quote Post
greynol
post Dec 31 2012, 17:43
Post #34





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I don't think this forum is scared of subjective opinions; we simply believe that expectation bias should not have a role at the table when it comes to gauging and expressing them.


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Dec 31 2012, 18:14
Post #35





Group: Members
Posts: 3647
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (jkauff @ Dec 29 2012, 09:18) *
QUOTE (Woodinville @ Dec 24 2012, 21:52) *
Many people have worked on measuring what you hear. No perceptual coder (MP3, AAC, AC3, etc) would work if many people had not only done that, but also done a good job of it.

My own experience makes me wonder how well human hearing has been measured. I'm sure average human hearing has been well-studied, but how many outliers are there in the world population? My daughter, for example, can hear the high frequency tones used by cat-repelling devices. She's demonstrated this many times, but my wife and I can't hear them at all (nor can most people).


Since you haven't given the actual operating frequency of this device, your claim lacks global meaning.

It is well known that children generally have better and more extended HF hearing than adults. Human ears have geometry that determines how they work and when they grow, the geometry shifts towards lower frequencies.

QUOTE
I wonder if the hearing range of various human populations has been well-studied (such as African tribes with certain members who can hear low frequency tones emitted by elephants).


We don't need to jump down every rabbit hole to know where to find rabbits every time we need to. ;-)

Perhaps one of the most cogent comments on this thread is from JJ, who makes the point that since lossy coders provide substantial amounts of data reduction with minimal sound quality loss all things considered, and they work based on what we know about human hearing, we must know quite a bit of useful information about human hearing.

QUOTE
Is it fair to say that this good work has been done primarily based on averages of 1st World populations?


More to the point there has been no need to develop special lossy encoders for every country, continent or people group. This is particularly true when good unbiased listening tests are used to determine what people are and are not hearing.
Go to the top of the page
+Quote Post
Ethan Winer
post Dec 31 2012, 18:14
Post #36





Group: Members
Posts: 248
Joined: 12-May 09
From: New Milford, CT
Member No.: 69730



QUOTE (pisymbol @ Dec 30 2012, 21:12) *
Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan


--------------------
I believe in Truth, Justice, and the Scientific Method
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Dec 31 2012, 18:25
Post #37





Group: Members
Posts: 3647
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (pisymbol @ Dec 30 2012, 21:12) *
Subjectivity can still provide valuable data points (especially negative impressions).


The above is like saying that radioactivity can lead to thermonuclear explosions.

Subjectivity can provide valuable data points, but the frequency of incidence of this happening can be disappointingly low.

Consider all of the high end audiophile publications. Thousands or tens of thousands of pages of subjective impressions per week or month that would simply cease to exist if proper listening test protocols were used.
Go to the top of the page
+Quote Post
Nessuno
post Dec 31 2012, 18:41
Post #38





Group: Members
Posts: 422
Joined: 16-December 10
From: Palermo
Member No.: 86562



QUOTE (Ethan Winer @ Dec 31 2012, 18:14) *
QUOTE (pisymbol @ Dec 30 2012, 21:12) *
Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan

This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.
What can really change and by large amounts are expectations in sighted tests, when the tester knows what he's listening to and could link to the listened device side effects of other precedent or contemporary experiences, different knowledge, true or supposed, even the mood of the moment...

This post has been edited by Nessuno: Dec 31 2012, 19:18


--------------------
... I live by long distance.
Go to the top of the page
+Quote Post
greynol
post Dec 31 2012, 20:29
Post #39





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Maybe we should take a close look at the meaning of the word perception. From where I stand, and from having an opinion shaped by what I have read by many of the fine members here and elsewhere with demonstrated expertise in the area, I will add that expectation bias does in fact alter our the perception of what we hear. The suggestion of a difference in X (breath/openness/dancability/PRAT but also non-BS qualities) can result in the actual and very real perception of that difference in X to the individual, even when the stimulus is held constant.

So no, changing perception does not have to affect DBT results necessarily. In the case of having placebophiles with strong expectation bias we often see the reverse: DBT affects the perception, or maybe not until the results of the test are revealed. That's still no guarantee since some people think that getting 13 right out of 20 means they passed the test. Still others will continue to cling to their faith in fairy dust, especially when the perceived differences once again return when the constraint of a DBT has been lifted.

This post has been edited by greynol: Dec 31 2012, 22:09


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
jkauff
post Dec 31 2012, 21:28
Post #40





Group: Members (Donating)
Posts: 161
Joined: 1-October 01
From: Doylestown, PA
Member No.: 145



Next, on to wine tasting.... wink.gif
Go to the top of the page
+Quote Post
greynol
post Dec 31 2012, 22:13
Post #41





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Feel free to start a new topic, replacing the word hear in the subject line with the word taste. Be sure to post it in the off-topic forum. wink.gif

This post has been edited by greynol: Dec 31 2012, 22:14


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
Woodinville
post Jan 1 2013, 03:06
Post #42





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (greynol @ Dec 31 2012, 08:43) *
I don't think this forum is scared of subjective opinions; we simply believe that expectation bias should not have a role at the table when it comes to gauging and expressing them.



Amen to that. Sorry, but it had to be said twice.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post Jan 1 2013, 03:08
Post #43





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Martel @ Dec 31 2012, 07:15) *
What's the "best measuring" frequency response of a headphone? I wonder... smile.gif


On whose head, with what haircut, under what circumstances, with what ear canal volume and shape?

Martel has a real point here. smile.gif


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post Jan 1 2013, 03:10
Post #44





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Nessuno @ Dec 31 2012, 09:41) *
This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.


To some extent that's true.

Even something so simple as measuring one's hearing can be very different on day 1 vs. day 3 of getting off of the plane, or if you spent 8 hours on the show floor vs. ran through it with earplugs.

But this kind of thing can be managed, and that's also what controls are for.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post Jan 1 2013, 03:12
Post #45





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (greynol @ Dec 31 2012, 13:13) *
Feel free to start a new topic, replacing the word hear in the subject line with the word taste. Be sure to post it in the off-topic forum. wink.gif


Actually, I know of at least 1 or 2 good books on sensory evaluation of wine that directly read on audio testing if you think even a little bit. smile.gif


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
greynol
post Jan 1 2013, 03:25
Post #46





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



You're right, I just don't know that we should drop comparing/correlating audio measurements and actual hearing which we hardly touched upon in order to move onto expectation bias in wine tasting. If anyone should be suggesting the direction of the discussion it should be the original poster.


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
Nessuno
post Jan 1 2013, 12:47
Post #47





Group: Members
Posts: 422
Joined: 16-December 10
From: Palermo
Member No.: 86562



Maybe a little OT, but I recall a "rule of thumb" more than a woman told me they follow when they buy perfumes is they don't evaluate more than three or even two different fragrances a time, or they would not be able to discern real differences and end up taking home something they could even not like any more days after.
I guess this has everything to do with temporal masking and even perception system fatigue.

P.S. Happy new year full of good music to all HA fellows! smile.gif


--------------------
... I live by long distance.
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Jan 1 2013, 15:47
Post #48





Group: Members
Posts: 3647
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (Woodinville @ Dec 31 2012, 21:12) *
QUOTE (greynol @ Dec 31 2012, 13:13) *
Feel free to start a new topic, replacing the word hear in the subject line with the word taste. Be sure to post it in the off-topic forum. wink.gif


Actually, I know of at least 1 or 2 good books on sensory evaluation of wine that directly read on audio testing if you think even a little bit. smile.gif


I'm reminded of Mielgard, et al classic book about Sensory Evaluation.

http://books.google.com/books?id=F_A-YtWXF3gC
Go to the top of the page
+Quote Post
Ethan Winer
post Jan 1 2013, 19:09
Post #49





Group: Members
Posts: 248
Joined: 12-May 09
From: New Milford, CT
Member No.: 69730



QUOTE (Nessuno @ Dec 31 2012, 12:41) *
but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.


There are several different issues. One is that our perception changes from moment to moment. Another is we can't focus on everything all at once when complex music is playing. Another is that some things really are so obvious that they won't be missed whether sighted or blind. Another is expectation affecting our opinion when we know which [device] we're hearing. Another is that moving your head even one inch in any direction really does change the sound reaching your ears (speakers, not headphones). Another is that our sensitivity changes at, as you said, borderline conditions. A real change may be noticed one time but not another. Especially for an old man like me. laugh.gif

--Ethan

This post has been edited by Ethan Winer: Jan 1 2013, 19:10


--------------------
I believe in Truth, Justice, and the Scientific Method
Go to the top of the page
+Quote Post
Martel
post Jan 1 2013, 19:54
Post #50





Group: Members
Posts: 553
Joined: 31-May 04
From: Czech Rep.
Member No.: 14430



QUOTE (Ethan Winer @ Jan 1 2013, 20:09) *
... moving your head even one inch in any direction really does change the sound reaching your ears (speakers, not headphones)
Almost all headphones "suffer" from a similar problem - seating them on/in one's head/ear (or any headphone measurement device). Two people are not very likely to position the same headphone exactly the same way on their heads due to different head/ear shapes, preferences etc. This is going to change the perceived/measured sound and introduce additional variable(s) into the mix.


--------------------
IE4 Rockbox Clip+ AAC@192; HD 668B/HD 518 Xonar DX FB2k FLAC;
Go to the top of the page
+Quote Post

4 Pages V  < 1 2 3 4 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th July 2014 - 19:06