IPB

Welcome Guest ( Log In | Register )

Why Live-vs-Recorded Listening Tests Don't Work
solive
post Jul 10 2010, 11:08
Post #1





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



Thomas Edison was probably the greatest stereo salesman that ever lived. He believed that "listeners will hear what you tell them to hear", and he was pretty successful convincing thousands of listeners that his 1910 Edison Diamond Disc Phonograph reproduced recordings that sounded identical to a live performance. His secret weapon was an elaborate live-versus-recorded demonstration that managed to convince people that his phonograph sounded a lot better than it really was.

Several times over the past 10 years, I have been asked by live-versus-recorded apologists why I don't do these types of the tests since they claim they are the only true valid measures of loudspeaker fidelity or accuracy. That is what prompted me to write about why I believe live-versus-recorded listening tests don't work, in this month's blog article

Cheers
Sean Olive
Audio Musings

This post has been edited by solive: Jul 10 2010, 11:30


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
 
Start new topic
Replies
kdo
post Jul 10 2010, 16:53
Post #2





Group: Members (Donating)
Posts: 304
Joined: 18-April 02
From: Russia
Member No.: 1812



I have a couple of questions.

QUOTE
Live and Recorded Performances Must Be Identical

For live-versus-recorded tests to be valid, the live and recorded performance should be identical, having the same notes, intonation, tempo, dynamics, loudness, balance between instruments, and the same location and sense of space of the instruments. Otherwise, there are extraneous cues that allow listeners to readily identify the live and recorded performances. Midi-controlled instruments (e.g. player pianos) are but one example of how this problem could be resolved.


Would it be possible to design a valid test with the opposite approach? That is, instead of trying to reproduce a single performance identically, could we use various different performances and recordings every time?

I'm thinking of such scenario: suppose we need a test with 20 trials. Take 20 different singers with different voices, make 20 recordings. And then let some 20 more singers (again, all different voices) perform during the test, a sort of A/B test.
This way, I'm thinking, the singers don't even have to perform the same piece of music. It could be different music material every trial/performance.

Would it be possible to gather any statistically significant result from such a test?



And my 2nd question: can we consider our everyday practice of enjoying recorded music as the "ultimate" proof that such recordings are indeed capable of creating an illusion of live performance?
After several decades of such practical experience all across the globe, perhaps we already have enough evidence to draw some statistically valid conclusions? or still not?
I mean, okay, measuring the accuracy of a particular loudspeaker is one thing, but can't we say anything definitive of the technology in general?

This post has been edited by kdo: Jul 10 2010, 16:55
Go to the top of the page
+Quote Post
solive
post Jul 12 2010, 06:46
Post #3





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (kdo @ Jul 10 2010, 08:53) *
I have a couple of questions.

QUOTE
Live and Recorded Performances Must Be Identical

For live-versus-recorded tests to be valid, the live and recorded performance should be identical, having the same notes, intonation, tempo, dynamics, loudness, balance between instruments, and the same location and sense of space of the instruments. Otherwise, there are extraneous cues that allow listeners to readily identify the live and recorded performances. Midi-controlled instruments (e.g. player pianos) are but one example of how this problem could be resolved.


Would it be possible to design a valid test with the opposite approach? That is, instead of trying to reproduce a single performance identically, could we use various different performances and recordings every time?

I'm thinking of such scenario: suppose we need a test with 20 trials. Take 20 different singers with different voices, make 20 recordings. And then let some 20 more singers (again, all different voices) perform during the test, a sort of A/B test.
This way, I'm thinking, the singers don't even have to perform the same piece of music. It could be different music material every trial/performance.

Would it be possible to gather any statistically significant result from such a test?



And my 2nd question: can we consider our everyday practice of enjoying recorded music as the "ultimate" proof that such recordings are indeed capable of creating an illusion of live performance?
After several decades of such practical experience all across the globe, perhaps we already have enough evidence to draw some statistically valid conclusions? or still not?
I mean, okay, measuring the accuracy of a particular loudspeaker is one thing, but can't we say anything definitive of the technology in general?


1) I think a basic tenet of a good scientific experiment is that it is repeatable. So using humans musicians as sound sources is going to cause a lot of errors, and biases if the live performance doesn't perfectly match the recorded one. If you can devise a way to compare the live performance (via live mic feeds w. no delay) and compare that double-blind to the performance that would eliminate some of the errors.

I don't see how your method gets around this problem. You can have 20 singers but unless their performances perfectly match their recordings then listeners have extraneous cues besides sound quality that tell them something is different.

2) I think it has been proven that most people can pretty well enjoy music listening to any old piece of crap. I first became really aware of how sound quality affects my enjoyment of music when someone recorded my piano recital with 2 mics located underneath the piano, and charged me for the tape. It didn't sound at all like how I played the piano (really boomy and dull). I was really pissed off. At that point I realized the importance of sound recording and reproduction and decided to pursue it as a career. But most people probably never think about it, until they go to a really bad live concert, and the artist doesn't sound anything like the recordings. Even then, quantity usually matters more than quality (think rock n'roll).

Cheers
Sean Olive
Audio Musings

This post has been edited by solive: Jul 12 2010, 06:47


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post

Posts in this topic
- solive   Why Live-vs-Recorded Listening Tests Don't Work   Jul 10 2010, 11:08
- - kdo   I have a couple of questions. QUOTE Live and Reco...   Jul 10 2010, 16:53
|- - kdo   QUOTE (kdo @ Jul 10 2010, 17:53) Would it...   Jul 11 2010, 16:25
||- - solive   QUOTE (kdo @ Jul 11 2010, 08:25) QUOTE (k...   Jul 13 2010, 00:43
|||- - kdo   QUOTE (solive @ Jul 13 2010, 01:43) The p...   Jul 13 2010, 01:02
||||- - solive   QUOTE (kdo @ Jul 12 2010, 17:02) QUOTE (s...   Jul 13 2010, 04:09
|||- - Arnold B. Krueger   QUOTE (solive @ Jul 12 2010, 19:43) It is...   Jul 13 2010, 20:26
||- - solive   QUOTE (kdo @ Jul 11 2010, 08:25) QUOTE (k...   Jul 13 2010, 04:53
||- - kdo   QUOTE (solive @ Jul 13 2010, 05:09) Sorry...   Jul 13 2010, 20:10
|- - solive   QUOTE (kdo @ Jul 10 2010, 08:53) I have a...   Jul 12 2010, 06:46
|- - kdo   QUOTE (solive @ Jul 12 2010, 07:46) 1) I ...   Jul 12 2010, 21:25
|- - Arnold B. Krueger   QUOTE (solive @ Jul 12 2010, 01:46) 1) I ...   Jul 13 2010, 17:25
|- - zane9   QUOTE (Arnold B. Krueger @ Jul 13 2010, 11...   Jul 13 2010, 18:50
||- - Arnold B. Krueger   QUOTE (zane9 @ Jul 13 2010, 13:50) QUOTE ...   Jul 13 2010, 20:21
||- - kdo   I sense a big fat TOS-8 violation right here: QUOT...   Jul 14 2010, 02:19
||- - Ed Seedhouse   One form of "live vs. reccorded" test wi...   Jul 14 2010, 02:35
|||- - kdo   QUOTE (Ed Seedhouse @ Jul 14 2010, 03:35)...   Jul 14 2010, 02:57
||- - Arnold B. Krueger   QUOTE (kdo @ Jul 13 2010, 21:19) I sense ...   Jul 14 2010, 03:12
||- - kdo   QUOTE (Arnold B. Krueger @ Jul 14 2010, 04...   Jul 14 2010, 03:30
||- - Arnold B. Krueger   QUOTE (kdo @ Jul 13 2010, 22:30) QUOTE (A...   Jul 14 2010, 10:42
||- - kdo   QUOTE (Arnold B. Krueger @ Jul 14 2010, 11...   Jul 14 2010, 16:56
||- - Arnold B. Krueger   QUOTE (kdo @ Jul 14 2010, 11:56) QUOTE (A...   Jul 14 2010, 17:15
||- - kdo   QUOTE (Arnold B. Krueger @ Jul 14 2010, 18...   Jul 14 2010, 17:58
|- - 2Bdecided   QUOTE (Arnold B. Krueger @ Jul 13 2010, 17...   Jul 14 2010, 19:04
|- - analog scott   QUOTE (2Bdecided @ Jul 14 2010, 19:04) QU...   Jul 14 2010, 20:40
|- - Arnold B. Krueger   QUOTE (2Bdecided @ Jul 14 2010, 14:04) It...   Jul 16 2010, 07:58
|- - 2Bdecided   QUOTE (Arnold B. Krueger @ Jul 16 2010, 07...   Jul 16 2010, 16:29
|- - Arnold B. Krueger   QUOTE (2Bdecided @ Jul 16 2010, 11:29) Cr...   Jul 16 2010, 23:59
- - analog scott   QUOTE (solive @ Jul 10 2010, 12:08) Thoma...   Jul 10 2010, 18:41
|- - solive   QUOTE (analog scott @ Jul 10 2010, 10:41)...   Jul 11 2010, 07:46
|- - analog scott   QUOTE (solive @ Jul 11 2010, 07:46) What ...   Jul 11 2010, 15:50
|- - googlebot   QUOTE (analog scott @ Jul 11 2010, 16:50)...   Jul 11 2010, 16:53
||- - analog scott   QUOTE (googlebot @ Jul 11 2010, 16:53) QU...   Jul 11 2010, 17:44
||- - aclo   QUOTE (googlebot @ Jul 11 2010, 17:53) Wh...   Jul 12 2010, 04:06
|- - solive   QUOTE (analog scott @ Jul 11 2010, 07:50)...   Jul 11 2010, 20:16
||- - analog scott   QUOTE (solive @ Jul 11 2010, 21:16) QUOTE...   Jul 11 2010, 20:33
||- - googlebot   QUOTE (solive @ Jul 11 2010, 21:16) The r...   Jul 11 2010, 20:33
|||- - solive   QUOTE (googlebot @ Jul 11 2010, 12:33) QU...   Jul 12 2010, 06:19
||- - 2Bdecided   QUOTE (solive @ Jul 11 2010, 20:16) The r...   Jul 14 2010, 18:54
|- - greynol   QUOTE (analog scott @ Jul 11 2010, 07:50)...   Jul 11 2010, 20:28
|- - Notat   QUOTE (greynol @ Jul 11 2010, 13:28) It...   Jul 11 2010, 23:06
- - greynol   Quite unfortunate if someone wants to garner somet...   Jul 11 2010, 23:19
- - greynol   Unless he can assure us that his opinion about the...   Jul 12 2010, 02:26
- - greynol   analogscott's post binned per TOS #2. Further...   Jul 12 2010, 03:09


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th September 2014 - 21:28