IPB

Welcome Guest ( Log In | Register )

24/96 digitalization - can it be audible, blind test results
pawelq
post Jun 18 2011, 23:14
Post #1





Group: Members
Posts: 541
Joined: 20-December 05
From: Springfield, VA
Member No.: 26522



I would like to share results of a blind listening test that was conducted on June 16th, 2011 in Warsaw, Poland.

The full description of the test is available at http://www.audio.e-snp.net/, but it’s in Polish, so I’ll provide a synopsis below.

The test question was "can we hear any effect of A/D/A conversion at 24/96 performed with a studio recorder". Frankly, both the test organizer and me (let's say, the test statistician), were pretty much convinced that no difference would be detectable.

The test music was played using a gramophone (Bergman Audio Sindre with Air Tight PC-1 cartridge), gramophone preamp (Air Tight ATE-2005), * , amplifier (Soulution 720 / 710), and loudspeakers (Hansen Audio Prince V2). In “analog” trials, the path was as described above. In “digital” trials, a studio recorder (Tascam DV-RA1000) set into “monitor” mode (i.e., A/D then D/A, at 24 bit and 96 kHz) was inserted in place of the asterisk.

The participants listened to about 1 minute of “Falling Alice” from Chick Corea’s “Mad Hatter” LP from 1976, mint condition. The test was conducted in a 5.8 m x 7.8 m acoustically adapted room.

There were 10 listeners (neither the test organizer nor me participated; he was switching the connections, I was several thousand kilometers away). All listeners listened together, being in the same room. They left the room for connection switching. During listening, the test organizer remained in the back of the room, invisible to the listeners. The listeners are (and me and the organizers) are members of a small Polish “sensible audiophile” internet forum.

There were 13 test trials, in 7 seven of them there was A/D/A conversion (D), in 6 purely analog path was used (A). The order of D and A trials was random.

Prior to the test, the listeners familiarized themselves with the supposed difference in A vs. D sound and with the recording, which was played a few times in A and D configuration. The listeners received answer cards on which they marked the trials as A or D. They were asked to answer in each trial, even if they were unsure. Prior to the test, they also provided answers to three questions: “Do you think that the effect of digitalization will be audible (yes/no)?”, “Do you consider yourself an experienced listener of vinyl records, using high-quality equipment (yes/no)?”, “How much of your listening time is spent listening to vinyl records (in %)”.

The results were analyzed in two main ways.
In the individual analysis, we checked if any of the participants identified A vs. D at a statistically significant level. With one-way binomial test with Šidák correction (due to multiple listeners. i.e., multiple tests) we determined that at most one error (12/13 correct) is allowed to pass this test (p=0.017; for 11/13 p=0.107).
In group analysis, we converted the results to proportion correct (e.g., 8/13=0.615) and used one-way Wilcoxon one-sample test to determine if the median of proportion correct was significantly higher than 0.5. (Additionally, we calculated one-way one-sample t-test, however, due to the small sample size, the normality assumption could not be reliably tested and we consider the results of t-test to be less trustworthy.)

The results table, a plot of proportion correct for each listener, and a more detailed description of the analysis are provided at http://www.audio.e-snp.net/wyniki.php (FYI, in the result table A stands for “analog” trials, C stands for “digital” trials, column 1 is listener number, columns 2-4 are answers to the three questions (NIE=no, TAK=yes), the bottom row shows what actually happened in the trials.)
No listener answered with 0 or 1 error, which was required for statistically significant outcome of the individual analysis. There was one 11/13 (0.846) result, and two 10/13 (0.769) results. No one scored below 6/13 (0.462).

The interesting thing is, that average proportion correct was 0.631, and it was significantly higher than chance (p=0.0322, Wilcoxon test; possibly unreliable t-test: p=0.0093). My interpretation is that the A/D/A process done with the Tascam recorder did audibly influence the signal.
No association of answers to any of the three questions with the proportion of correct answers was found.

Any comments?



--------------------
Ceterum censeo, there should be an "%is_stop_after_current%".
Go to the top of the page
+Quote Post
 
Start new topic
Replies
usernaim
post Jun 20 2011, 16:21
Post #2





Group: Members
Posts: 61
Joined: 4-May 08
Member No.: 53291



Interesting that no one did worse than 6/13. Just by intuition I find that to be potentially informative.

But the multiple listeners issue is bothersome. Is there any possibility that the listeners influenced each other?

Also, as far as applicability, we don't tend to listen to our hi-fi that way, with all that acoustic interference. I don't know if that makes the test more or less difficult or doesn't matter, but it might matter.
Go to the top of the page
+Quote Post
[JAZ]
post Jun 20 2011, 18:41
Post #3





Group: Members
Posts: 1797
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



QUOTE (usernaim @ Jun 20 2011, 17:21) *
Interesting that no one did worse than 6/13. Just by intuition I find that to be potentially informative.


Doing 6/13 is not an indicative of being on the good track of things. 6/13 (and 7/13) is an indicative of chance. If you flip a coin, there's as much possibilities to get any of each sides. If you flip it twice, the randomness would suggest you get different sides each time, so achieveing a 1/2.

That's why here at hydrogenaudio we try to make people aware of ABX, and the way to understand the values ( asking to reach 95% or in some cases even 99% of success).

In fact, there is as much possibilities to reach 16/16 than to reach 0/16. 0/16, if not done by chance, would suggest that a difference was clearly noticed, but the user misinterpreted which was which when answering.


About the test itself, I am not knowledgeable enough to find what would make it incorrect, but having 3 out of 10 listeners with a result of 10/13 or better is something to try to understand. There is no proof (they didn't pass the test) that they could hear a difference, but the results imply that there might have been something.

This post has been edited by [JAZ]: Jun 20 2011, 18:43
Go to the top of the page
+Quote Post

Posts in this topic
- pawelq   24/96 digitalization - can it be audible   Jun 18 2011, 23:14
- - Notat   Did you carefully match the level of the system wi...   Jun 19 2011, 01:01
|- - pawelq   Thanks everyone for you comments. And sorry for m...   Jun 19 2011, 16:17
|- - Notat   QUOTE (pawelq @ Jun 19 2011, 09:17) QUOTE...   Jun 20 2011, 00:14
|- - Parelius   QUOTE (pawelq @ Jun 19 2011, 17:17) We en...   Jun 20 2011, 20:25
|- - Notat   QUOTE (Parelius @ Jun 20 2011, 13:25) QUO...   Jun 20 2011, 23:09
|- - Parelius   ^ Sleeping in class. «overlook» should be «ignore»...   Jun 20 2011, 23:10
|- - WernerO   QUOTE (Parelius @ Jun 20 2011, 21:25) I...   Jun 21 2011, 07:03
- - Juha   By the specs, Tascam DV-RA1000 does not have very ...   Jun 19 2011, 07:23
|- - hlloyge   QUOTE (Juha @ Jun 19 2011, 07:23) By the ...   Jun 19 2011, 09:21
||- - Juha   QUOTE (hlloyge @ Jun 19 2011, 11:21) QUOT...   Jun 19 2011, 10:29
|- - WernerO   QUOTE (Juha @ Jun 19 2011, 07:23) By the ...   Jun 19 2011, 16:08
|- - Juha   QUOTE (WernerO @ Jun 19 2011, 18:08) QUOT...   Jun 19 2011, 20:13
|- - saratoga   QUOTE (Juha @ Jun 19 2011, 15:13) So, it...   Jun 20 2011, 00:24
||- - Northpack   QUOTE (saratoga @ Jun 19 2011, 23:24) QUO...   Jun 20 2011, 00:48
|- - WernerO   QUOTE (Juha @ Jun 19 2011, 20:13) isn...   Jun 20 2011, 11:18
- - DonP   Just throwing out some guesses here of things that...   Jun 19 2011, 11:50
- - krabapple   Just to be clear, this is all about an average sco...   Jun 20 2011, 03:55
|- - pawelq   QUOTE (krabapple @ Jun 19 2011, 22:55) Ju...   Jun 21 2011, 19:26
|- - 2Bdecided   QUOTE (pawelq @ Jun 21 2011, 19:26) QUOTE...   Jun 22 2011, 10:23
||- - pawelq   QUOTE (2Bdecided @ Jun 22 2011, 05:23) Th...   Jun 22 2011, 13:55
||- - Northpack   QUOTE (pawelq @ Jun 22 2011, 12:55) QUOTE...   Jun 22 2011, 14:39
||- - knutinh   QUOTE (pawelq @ Jun 22 2011, 14:55) QUOTE...   Jun 23 2011, 12:29
|- - Martel   QUOTE (pawelq @ Jun 21 2011, 20:26) I bel...   Jun 22 2011, 13:43
- - 2Bdecided   82/130. Assuming you didn't cherry pick the d...   Jun 20 2011, 10:11
- - usernaim   Interesting that no one did worse than 6/13. Just...   Jun 20 2011, 16:21
|- - [JAZ]   QUOTE (usernaim @ Jun 20 2011, 17:21) Int...   Jun 20 2011, 18:41
|- - usernaim   QUOTE ([JAZ] @ Jun 20 2011, 13:41...   Jun 20 2011, 19:46
- - AndyH-ha   In case it isn't clear, leaving headroom when ...   Jun 21 2011, 04:56
|- - Kees de Visser   QUOTE (AndyH-ha @ Jun 21 2011, 05:56...   Jun 21 2011, 09:46
- - AndyH-ha   What is the definition of a half of the input rang...   Jun 21 2011, 09:01
|- - WernerO   QUOTE (AndyH-ha @ Jun 21 2011, 10:01...   Jun 21 2011, 11:26
- - Northpack   I don't think that we have to speculate about ...   Jun 21 2011, 15:53
- - Martel   I'm no statistics expert but it seems like too...   Jun 21 2011, 17:26
- - Axon   FWIW, I recall once being able to successfully ABX...   Jun 22 2011, 16:24
- - Brand   FWIW, I just ABXed a song with a 0.1 dB difference...   Jun 23 2011, 13:38
|- - Northpack   QUOTE (Brand @ Jun 23 2011, 12:38) FWIW, ...   Jun 23 2011, 14:34
|- - Kees de Visser   QUOTE (Northpack @ Jun 23 2011, 15:34) An...   Jun 23 2011, 17:44
||- - Northpack   QUOTE (Kees de Visser @ Jun 23 2011, 16:4...   Jun 23 2011, 18:54
||- - dhromed   QUOTE (Kees de Visser @ Jun 23 2011, 18:4...   Jun 23 2011, 19:56
|- - Brand   QUOTE (Northpack @ Jun 23 2011, 15:34) QU...   Jun 24 2011, 12:56
- - pawelq   Just a question. Can you guys replicate this kind ...   Jun 24 2011, 13:03
- - Brand   I haven't tried yet, so I won't make any c...   Jun 24 2011, 13:54
|- - pawelq   QUOTE (Brand @ Jun 24 2011, 08:54) But si...   Jun 25 2011, 05:48
- - Brand   I previously kinda implied that using headphones m...   Jun 27 2011, 22:33
- - Axon   :F If it's easier with speakers, then I'm...   Jun 28 2011, 21:33
|- - DonP   QUOTE (Axon @ Jun 28 2011, 15:33) :F If ...   Jun 28 2011, 23:44
|- - Axon   QUOTE (DonP @ Jun 28 2011, 17:44) QUOTE (...   Jul 13 2011, 01:57
- - Dynamic   I have a couple of queries regarding methodology (...   Jun 30 2011, 00:14
- - Arnold B. Krueger   QUOTE (pawelq @ Jun 18 2011, 18:14) There...   Jun 30 2011, 12:18
- - RonaldDumsfeld   Is there a possibility that some convertor chips a...   Jul 1 2011, 23:29
|- - Arnold B. Krueger   QUOTE (RonaldDumsfeld @ Jul 1 2011, 18:29...   Jul 13 2011, 20:04
- - 2Bdecided   If one of the speaker cones has a hard physical re...   Jul 13 2011, 12:58
- - Axon   The best idea I could come up with is if the disto...   Jul 13 2011, 16:37
- - knutinh   So what is the conclusion on the topic at hand? -...   Jul 13 2011, 21:48
- - AndyH-ha   My apologies for not having the energy to read eno...   Jul 14 2011, 06:59


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th December 2014 - 23:03