IPB

Welcome Guest ( Log In | Register )

44.1 vs 88.2 ABX report at AES
2Bdecided
post Jul 16 2010, 17:07
Post #1


ReplayGain developer


Group: Developer
Posts: 5138
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Here:

http://www.aes.org/events/128/papers/?ID=2252

QUOTE
P18-6 Sampling Rate Discrimination: 44.1 kHz vs. 88.2 kHz—Amandine Pras, Catherine Guastavino, McGill University - Montreal, Quebec, Canada
It is currently common practice for sound engineers to record digital music using high-resolution formats, and then down sample the files to 44.1 kHz for commercial release. This study aims at investigating whether listeners can perceive differences between musical files recorded at 44.1 kHz and 88.2 kHz with the same analog chain and type of AD-converter. Sixteen expert listeners were asked to compare 3 versions (44.1 kHz, 88.2 kHz, and the 88.2 kHz version down-sampled to 44.1 kHz) of 5 musical excerpts in a blind ABX task. Overall, participants were able to discriminate between files recorded at 88.2 kHz and their 44.1 kHz down-sampled version. Furthermore, for the orchestral excerpt, they were able to discriminate between files recorded at 88.2 kHz and files recorded at 44.1 kHz.
Convention Paper 8101


Was anyone at the presentation? Has anyone bought the paper?

Cheers,
David.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
krabapple
post Jul 16 2010, 22:23
Post #2





Group: Members
Posts: 2274
Joined: 18-December 03
Member No.: 10538



1) I suggest this thread be focused on the 44.1 vs 88.2 paper. A separate thread can be about the MP3 study.

2) I bought the paper. Here's a paraphrase of the methods and results. Note that the test signals were recorded by the authors.


subjects : 15 male, 1 female, all having at least 3 yrs of sound engineering experience, six being pros, ten being students. All but one were musically trained.


equipment: the recording microphones (a pair of Sennheiser MKH 8020) had a FR of 10Hz-60kHz. Two stereo feeds from the mic preamp (Millennia HV-3D) to two Micstasy ADCs, one set to 44.1/24 the other to 88.2/24 ; then the 44.1/24 digital signal was recorded (at 44.1) on a Sound Devices 744T portable recorder, while the 88.2 output was recorded on a MacBook Pro at 88.2 using Logic Studio software. The recording diagram also shows that the 44.1 ADC used its internal clock, while the 88.2 ADC's master clock was a Mutec .


test signals: five musical/instrumental (orchestra, classical guitar, cymbals , voice, violin) recordings by the authors, from live performances at McGill that took place in several halls/rooms with varying dimensions & acoustics. For use in tests, 5-8 sec excerpts were used, with no processing except fade in.out via Pyramix 6 software. Care was taken to make the fades the same on the 44.1 and 88.2 examples. The 88.2 excerpts were also then downsampled to 44.1 via Pyramix software, so there were three sets of signals, native 44.1, native 88.2, and downsampled 88.2-->44.1.


playback: 5 blocks corresponding to the 5 excerpts, 12 trials per block ( i.e. all pairwise combinations of the three versions, each presented 4 times, twice in each of the two presentation orders) . Randomized ABX protocol was used. Listening occurred in an ITU standard room (the Critical Listening Lab of the CIRMMT, Montreal). Plaback hardware was an RME Fireface 800 DAC, a Grace m906 monitor controller, and a Classe CA-5200 stereo amp, feeding a pair of B&W 802D loudspeakers (FR 70Hz-33kHz). The authors picked the Fireface because 'it was the only converter that allowed us to switch sample rates between 44.1 and 88.2 in a respectable amount of time." To avoid clipping a 750ms switching interval was employed, set in the user interface which was Cycle '74's Max/MSP/Jitter software package. (I'm not quite clear from this how ABX switching was done, though I'm guessing the UI allows it?). All playback was at 24 bits.


Results: Considering cumulative binomial test results ( i.e., for all comparisons of all excerpts), 3/16 individuals achieved significant results (p < 0.05, 2-tailed ) but 'they significantly selected the wrong answer' (?!). The other 13 didn't perform better than chance, either individually or as a group.
But for these 13 subjects as a group, IF one considers each format comparison separately (rather than combining all comparisons) significant results were observed for 88.2 vs downsampled (p = .04, 2-tailed) . For the same group a 'tendency' was observed for 88.2 vs native 44.1 (p = 0.1) . No significant diff between native 44.1 and downsampled 44.1 (p = .2, I guess this doesn't constitute a 'tendency').
Results for the 13 subjects (grouped) were also re-analysed by musical type. Significant: Orchestral excerpt for 88.2 vs native 44.1 (p=.02). Classical Guitar and Voice excerpts for 88.2 vs downsampled (p = .004, p= .04). Not significant for any excerpt: 44.1 vs downsampled 44.1

The results of the three subjects who signficantly picked the wrong answer were also analyzed further, on a by-format basis (presumably the results of the three subjects were grouped). Significant: 88.2 vs native 44.1 (p = 0.02); native 44.1 vs downsampled 44.1 (p = 0.02). Not significant: 88.2 vs native 44.1 (p = .15). On a musical type basis, significant: Violin excerpt for 88.2 vs 44.1 (p = .006); Guitar and Violin excerpts, 44.1 vs downsampled 44.1 (p = .02 , p = .006). (I presume these stats are all wrt RIGHT answers, not WRONG answers, in contrast to the cumulative results, but it's not clear to me if that's true).

Collapsed results of all 16 subjects showed significant different for 88.2 vs native 44.1 Orchestral excerpt (p =.01


All subjects reported (in post-test questionnaires) that it was a very demanding test and they had lots of doubts about their choices.


I won't summarize the discussion, where the authors try to explain some of the implications and curiosa of the test results. You should pay yer $20 for that!

This post has been edited by krabapple: Jul 16 2010, 22:24
Go to the top of the page
+Quote Post
mzil
post Jul 22 2012, 02:47
Post #3





Group: Members
Posts: 605
Joined: 5-August 07
Member No.: 45913



QUOTE (krabapple @ Jul 16 2010, 17:23) *
2) I bought the paper. Here's a paraphrase of the methods and results. Note that the test signals were recorded by the authors...

equipment: the recording microphones (a pair of Sennheiser MKH 8020) had a FR of 10Hz-60kHz. Two stereo feeds from the mic preamp (Millennia HV-3D) to two Micstasy ADCs, one set to 44.1/24 the other to 88.2/24 ; then the 44.1/24 digital signal was recorded (at 44.1) on a Sound Devices 744T portable recorder, while the 88.2 output was recorded on a MacBook Pro at 88.2 using Logic Studio software. The recording diagram also shows that the 44.1 ADC used its internal clock, while the 88.2 ADC's master clock was a Mutec .


test signals: five musical/instrumental (orchestra, classical guitar, cymbals , voice, violin) recordings by the authors, from live performances ...
[bold texting added by me]

Krabapple, do you still have the paper? [And are you still subscribed to this old thread and seeing this question, I wonder?] Do the authors make any mention of level matching (using instrumentation) for the stage I have indicated above in bold text?

They rather oddly decided to use live music as their test source, and not a high resolution recording as Arnold Krueger correctly mentions they "should have" (that is then manipulated to create the different competing signals), but what assurance do we have that the input stages of the two Micstasy ADCs (having variable levels of gain with both manual and "auto" modes, as I understand it), successfully recorded the two analog signals at exactly the same, precise level in the digital domain? If one was a fraction of a dB different than the other, that could be the difference listeners heard, right there! Furthermore, even if both machine's inputs were set to the exact same attenuation values, do we know for a fact that simply selecting a different sampling rate won't in itself alter the actual level of the digital signal, by a small amount?

I often see people in the subjective evaluation world naively assume that there's no need to introduce level matching when comparing, say, the output of two CD players, because "the spec sheets say they both have a fixed, 2.0V output", but in truth they often do vary (slightly) when measured using instrumentation, and that small level change could easily be the difference they are hearing (but often mistakenly attribute it to being one of "quality", not simply level). I am wonder if maybe this study suffered from the same sort of flaw [an assuption that levels are matched by default, yet they aren't]?

edit to add: Of course the playback chain also would need be tested for exactly matched output level. Just like CD players vary in output, despite almost all claiming "2.0V", DACs also may vary slightly depending on the sampling rate they receive.

This post has been edited by mzil: Jul 22 2012, 03:27
Go to the top of the page
+Quote Post

Posts in this topic
- 2Bdecided   44.1 vs 88.2 ABX report at AES   Jul 16 2010, 17:07
- - Dologan   Interesting. The fact that the discrimination was ...   Jul 16 2010, 18:18
|- - C.R.Helmrich   http://coltrane.music.mcgill.ca/MAQ/experiments co...   Jul 16 2010, 19:33
|- - Dologan   QUOTE (C.R.Helmrich @ Jul 16 2010, 19:33)...   Jul 16 2010, 20:30
- - Arnold B. Krueger   QUOTE (2Bdecided @ Jul 16 2010, 12:07) He...   Jul 16 2010, 19:45
|- - pbelkner   QUOTE (Arnold B. Krueger @ Jul 16 2010, 20...   Jul 16 2010, 20:01
||- - Dologan   QUOTE (pbelkner @ Jul 16 2010, 20:01) QUO...   Jul 16 2010, 20:03
||- - pbelkner   QUOTE (Dologan @ Jul 16 2010, 21:03) but ...   Jul 16 2010, 20:19
||- - benski   Yes, exactly. It is cheaper to design a 192kHz DA...   Jul 16 2010, 20:47
||- - Arnold B. Krueger   QUOTE (Dologan @ Jul 16 2010, 15:03) QUOT...   Jul 16 2010, 23:21
|- - C.R.Helmrich   QUOTE Overall, participants were able to discrimin...   Jul 16 2010, 20:25
|- - pbelkner   QUOTE (C.R.Helmrich @ Jul 16 2010, 21:25)...   Jul 16 2010, 20:59
- - krabapple   1) I suggest this thread be focused on the 44.1 v...   Jul 16 2010, 22:23
|- - WernerO   QUOTE (krabapple @ Jul 16 2010, 23:23) Th...   Aug 11 2010, 13:51
|- - mzil   QUOTE (krabapple @ Jul 16 2010, 17:23) 2)...   Jul 22 2012, 02:47
|- - C.R.Helmrich   Sorry for bumping this thread, but I thought the f...   Mar 8 2014, 01:21
|- - [JAZ]   QUOTE (C.R.Helmrich @ Mar 8 2014, 01:21) ...   Mar 8 2014, 18:38
- - googlebot   Why upsample? 99.9% of all DACs oversample anyway....   Jul 16 2010, 22:25
|- - Arnold B. Krueger   QUOTE (googlebot @ Jul 16 2010, 17:25) Wh...   Jul 16 2010, 23:47
|- - benski   QUOTE (Arnold B. Krueger @ Jul 16 2010, 18...   Jul 17 2010, 03:12
||- - Arnold B. Krueger   QUOTE (benski @ Jul 16 2010, 22:12) QUOTE...   Jul 17 2010, 06:39
||- - greynol   QUOTE (Arnold B. Krueger @ Jul 16 2010, 22...   Jul 17 2010, 18:33
||- - Arnold B. Krueger   QUOTE (greynol @ Jul 17 2010, 13:33) QUOT...   Jul 17 2010, 20:35
|- - 2Bdecided   QUOTE (Arnold B. Krueger @ Jul 16 2010, 23...   Jul 21 2010, 13:22
- - Alex B   I think the least incorrect way to compare 88.2 KH...   Jul 16 2010, 22:33
|- - Arnold B. Krueger   QUOTE (Alex B @ Jul 16 2010, 17:33) I thi...   Jul 16 2010, 23:30
|- - Dologan   QUOTE (Alex B @ Jul 16 2010, 22:33) I thi...   Jul 16 2010, 23:40
- - AndyH-ha   It seems that what I wrote in the first part of my...   Jul 17 2010, 09:55
|- - googlebot   AndyH-ha, that might be on purpose. For two lowpas...   Jul 17 2010, 10:49
- - googlebot   I think the 88.2 vs. downsampled 44.1 test had at ...   Jul 17 2010, 16:39
- - greynol   QUOTE (Arnold B. Krueger @ Jul 17 2010, 12...   Jul 17 2010, 20:48
|- - Arnold B. Krueger   QUOTE (greynol @ Jul 17 2010, 15:48) QUOT...   Jul 17 2010, 23:04
- - AndyH-ha   I also may not know what I'm talking about, bu...   Jul 17 2010, 22:02
- - greynol   The reason for oversampling in old CD players is a...   Jul 17 2010, 22:13
|- - Arnold B. Krueger   QUOTE (greynol @ Jul 17 2010, 17:13) The ...   Jul 17 2010, 23:14
- - greynol   I'm taking issue with what I thought was a bla...   Jul 17 2010, 23:27
|- - Arnold B. Krueger   QUOTE (greynol @ Jul 17 2010, 18:27) I...   Jul 18 2010, 01:52
|- - greynol   QUOTE (Arnold B. Krueger @ Jul 17 2010, 17...   Jul 19 2010, 00:56
- - amandinepras   Thanks all for your interest in our paper, I recei...   Jul 19 2010, 00:52
|- - Arnold B. Krueger   QUOTE (amandinepras @ Jul 18 2010, 19:52)...   Jul 19 2010, 03:35
|- - Arnold B. Krueger   QUOTE (amandinepras @ Jul 18 2010, 19:52)...   Jul 19 2010, 11:35
||- - 2Bdecided   QUOTE (Arnold B. Krueger @ Jul 19 2010, 11...   Jul 21 2010, 13:20
||- - Kees de Visser   QUOTE (2Bdecided @ Jul 21 2010, 14:20) QU...   Jul 23 2010, 09:26
|- - Arnold B. Krueger   QUOTE (amandinepras @ Jul 18 2010, 19:52)...   Jul 19 2010, 11:48
||- - googlebot   QUOTE (Arnold B. Krueger @ Jul 19 2010, 12...   Jul 19 2010, 12:19
|||- - Arnold B. Krueger   QUOTE (googlebot @ Jul 19 2010, 07:19) QU...   Jul 19 2010, 14:33
||- - krabapple   QUOTE (Arnold B. Krueger @ Jul 19 2010, 06...   Jul 19 2010, 14:08
|- - Notat   QUOTE (amandinepras @ Jul 18 2010, 17:52)...   Jul 21 2010, 04:08
|- - hciman77   QUOTE (amandinepras @ Jul 18 2010, 19:52)...   Jul 28 2010, 15:18
- - Pio2001   Thanks for joining the discussion, Amandine. Your...   Jul 19 2010, 12:31
|- - 2Bdecided   QUOTE (Pio2001 @ Jul 19 2010, 12:31) I se...   Jul 21 2010, 13:13
- - googlebot   The study shows at least the intent of objectivity...   Jul 19 2010, 16:28
|- - Arnold B. Krueger   QUOTE (googlebot @ Jul 19 2010, 11:28) [l...   Jul 19 2010, 19:11
|- - Soap   QUOTE (Arnold B. Krueger @ Jul 19 2010, 14...   Jul 19 2010, 20:24
|- - Soap   QUOTE (Arnold B. Krueger @ Jul 19 2010, 14...   Jul 19 2010, 23:07
- - Juha   QUOTE Doesn't anybody else see a problem with ...   Jul 19 2010, 19:25
- - googlebot   The manufacturer's specs for the FF 800. Unwei...   Jul 19 2010, 20:22
|- - Arnold B. Krueger   QUOTE (googlebot @ Jul 19 2010, 15:22) Th...   Jul 19 2010, 21:26
- - Cavaille   Forgive me to interrupt this discussion about the ...   Jul 20 2010, 08:40
|- - Arnold B. Krueger   QUOTE (Cavaille @ Jul 20 2010, 03:40) Arn...   Jul 20 2010, 13:13
|- - Arnold B. Krueger   QUOTE (Arnold B. Krueger @ Jul 20 2010, 08...   Jul 20 2010, 19:30
||- - Pio2001   QUOTE (Arnold B. Krueger @ Jul 20 2010, 20...   Jul 20 2010, 22:36
|||- - Arnold B. Krueger   QUOTE (Pio2001 @ Jul 20 2010, 17:36) QUOT...   Jul 20 2010, 23:57
||- - C.R.Helmrich   QUOTE (Arnold B. Krueger @ Jul 20 2010, 20...   Jul 20 2010, 22:53
|- - 2Bdecided   QUOTE (Arnold B. Krueger @ Jul 20 2010, 13...   Jul 21 2010, 13:25
|- - WernerO   QUOTE (Arnold B. Krueger @ Jul 20 2010, 14...   Aug 2 2010, 08:19
|- - Arnold B. Krueger   QUOTE (WernerO @ Aug 2 2010, 03:19) QUOTE...   Aug 2 2010, 11:56
- - Pio2001   The strange thing is that there are ABX results wi...   Jul 21 2010, 12:16
|- - hciman77   QUOTE (Pio2001 @ Jul 21 2010, 07:16) The ...   Jul 27 2010, 22:56
- - googlebot   The fact, that the study's authors have regist...   Jul 23 2010, 10:55
|- - Pio2001   QUOTE (googlebot @ Jul 23 2010, 11:55) co...   Jul 24 2010, 13:58
- - krabapple   Sheesh, it's only been four days. They may ac...   Jul 23 2010, 19:20
- - googlebot   In dubio pro reo is generally a good principle. Bu...   Jul 26 2010, 19:43
|- - hciman77   QUOTE (googlebot @ Jul 26 2010, 14:43) In...   Jul 26 2010, 21:05
- - hciman77   I read the full paper and I think there may be som...   Jul 26 2010, 19:48
|- - Arnold B. Krueger   QUOTE (hciman77 @ Jul 26 2010, 14:48) I r...   Jul 27 2010, 13:16
||- - 2Bdecided   QUOTE (Arnold B. Krueger @ Jul 27 2010, 13...   Jul 27 2010, 22:45
|- - Pio2001   QUOTE (hciman77 @ Jul 26 2010, 20:48) Thi...   Jul 28 2010, 11:39
|- - Arnold B. Krueger   QUOTE (Pio2001 @ Jul 28 2010, 06:39) QUOT...   Jul 28 2010, 13:52
|- - AndyH-ha   QUOTE (Arnold B. Krueger @ Jul 28 2010, 04...   Jul 28 2010, 22:42
|- - Arnold B. Krueger   QUOTE (AndyH-ha @ Jul 28 2010, 17:42...   Jul 29 2010, 13:04
|- - 2Bdecided   QUOTE (AndyH-ha @ Jul 28 2010, 22:42...   Jul 29 2010, 22:11
|- - SebastianG   QUOTE (2Bdecided @ Jul 29 2010, 22:11) bu...   Jul 30 2010, 09:55
|- - Arnold B. Krueger   QUOTE (2Bdecided @ Jul 29 2010, 17:11) I ...   Jul 30 2010, 12:26
- - lvqcl   Audition 1.5: 44.1 -> 96 kHz resampling, Qualit...   Jul 30 2010, 11:42
- - C.R.Helmrich   I finally found time to read the entire paper. It...   Aug 9 2010, 21:41
- - 2Bdecided   I think before you pull out one positive result an...   Aug 9 2010, 22:44
|- - C.R.Helmrich   QUOTE (2Bdecided @ Aug 9 2010, 23:44) I t...   Aug 10 2010, 01:37
- - Pio2001   I finaly got the article. Actually, they say somet...   Aug 9 2010, 23:33
- - Pio2001   The unknown thing is the origin of the p values. I...   Aug 10 2010, 12:33
- - lrossouw   Did they test for difference (Check if you can tel...   Sep 8 2010, 09:39
- - Mach-X   I am going to loosely quote a well known poster he...   Mar 8 2014, 08:08
|- - Arnold B. Krueger   QUOTE (Mach-X @ Mar 8 2014, 03:08) I...   Mar 10 2014, 12:58
- - bandpass   TEAC -- slight COI perhaps? Guessing their method...   Mar 8 2014, 09:06
- - WernerO   Perhaps. Another weakness is their apparent use o...   Mar 11 2014, 08:04
|- - Arnold B. Krueger   QUOTE (WernerO @ Mar 11 2014, 03:04) Perh...   Mar 11 2014, 13:44
||- - Kees de Visser   QUOTE (Arnold B. Krueger @ Mar 11 2014, 13...   Mar 11 2014, 14:28
|- - C.R.Helmrich   QUOTE (WernerO @ Mar 11 2014, 08:04) Anot...   Mar 11 2014, 23:55
- - Wombat   QUOTE (Kees de Visser @ Mar 11 2014, 14:2...   Mar 11 2014, 15:51
2 Pages V   1 2 >


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th September 2014 - 18:59