IPB

Welcome Guest ( Log In | Register )

AES conference London: High Resolution perception, paper about listening test
Kees de Visser
post Jul 5 2007, 11:25
Post #1





Group: Members
Posts: 676
Joined: 22-May 05
From: France
Member No.: 22220



Unfortunately I wasn't able to visit the june 2007 AES Conference in London about High Resolution Audio.
The paper/presentation about a high-res audio listening test seems interesting. I'm wondering if anyone on HA happens to have been there and can share some information.

preview of the paper session:
QUOTE
Monday, June 25 11:00 – 12:30
Paper Session 2 — Perception

2-1 Which of the Two Digital Audio Systems Meets Best with the Analog System?— Wieslaw Woszczyk,1 Jan Engel,2 John Usher,1 Ronald Aarts,3 Derk Reefman3
1McGill University, Montreal, Quebec, Canada
2Centre for Quantitative Methods CQM BV
3Philips Research, Eindhoven, The Netherlands

In this listening test, two digital audio systems (B and C), and one analog system (A) were tested by 10 test persons who listened to a surround sound scene “live” (without recording). The main question to be answered was: “Which of the two digital systems meets best with the analog system?” Both digital versions had 24-bit dynamic resolution but differed in sampling rate with which the analog signal was sampled. One version © was sampled with a CD rate of 44.1 kHz, the other (B) 8 times faster. There were also two test conditions, where in one condition there was a bandwidth cut off at 20 kHz instead of the 100 kHz that was possible with special 100 kHz microphones and added super-tweeters. For each subject, the experiment was replicated six times, in each of the two conditions. The outcome of each experiment was a 0 or 1, where the 1 means that the, technically best, digital system B has been chosen as meeting the analog quality. The paper describes the test and the outcome.

Without having read the paper, it's not clear to me whether the test was double-blind or not. Apparently it was not possible to replay sources, since the audio source was "live". How reliable would a test like this be ?
Go to the top of the page
+Quote Post
 
Start new topic
Replies
boombaard
post Dec 24 2007, 12:02
Post #2





Group: Members
Posts: 336
Joined: 7-February 05
From: Local Cluster
Member No.: 19647



must've missed it when you posted it first.. odd.

anyway

QUOTE
However, to achieve a higher degree of fidelity to the live analog reference, we need to convert audio using high sampling rate even when we do not use microphones and loudspeakers having bandwidth extended far beyond 20 kHz. Listeners judge high sampling conversion as sounding more like the analog reference when listening to standard audio bandwidth.


and this quote struck me as very, very odd.
(read the article first, or at least the conclusions/test setup bits before posting.. the phrasing seems to be a bit awkward in places, but i can't really help that)

because unless i'm really, really reading this wrong they're saying that if you upsample to the highest SR your sound card can take before playing it back - no matter the quality of the input material - it will be perceived as 'more natural'.

(mind you, i don't really see why you'd need to do what they suggest next, namely:
QUOTE
These results suggest that the archiving community should consider using high-sampling conversion to ensure transparency even if the recording is made with standard audio-bandwidth transducers, and when digitizing older recordings made with bandwidth-limited analog systems.
just turning on SSRC in your foobar dsp chain would seem to be enough)

the point, really, is that i almost can't believe that i'm understanding this right.
A second point: even if they are correct, it would seem that their conclusion that anything >20kHz is a waste of bw is somewhat hasty, and they imo should've added a 'middle' test - to say 40kHz - just to narrow down where the added bandwidth became overkill (more).

QUOTE (Kees de Visser @ Dec 24 2007, 09:04) *
PS:
QUOTE
because unless i'm really, really reading this wrong they're saying that if you upsample to the highest SR your sound card can take before playing it back - no matter the quality of the input material - it will be perceived as 'more natural'.
No, it's not about upsampling. The test compares an AD/DA chain to the analog source at two differents sample rates: 44.1 and 352.8 kHz. This is basically different from upsampling 44.1 kHz data that already have passed a low pass filter.


well, the topic of the paper is at least remarkably on-topic.. but i don't really see what you base your statement about the AD/DA chain on.
as i understand it, both the 20kHz recording equip and the 100kHz ones were attached to the same chain, so that doesn't really seem relevant here.
furthermore:
QUOTE
In the 20 kHz case (C2), they liked the high-sampling version, which would have an effect on other aspects of technical quality than the high frequency response. This may indicate that we do not need 100 kHz microphones and 100 kHz loudspeakers, but we do need 100 kHz capable recorders.


so what are they saying here then? that 20kHz cutoff mikes still produce overtones/noise in the 20-100kHz band that's relevant to the recording and is perceived as adding to the perceived 'naturalness' of said recording, even though they're not 'real'?

anyway, what i don't really see is why that would influence the archiving community.. i can see how this might be interesting to recording studios, but by the time it reaches the consumer, the format is already decided on. so why would it be relevant to him to save it in as high a sampling rate as possible?
Go to the top of the page
+Quote Post
Kees de Visser
post Dec 24 2007, 15:32
Post #3





Group: Members
Posts: 676
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (boombaard @ Dec 24 2007, 12:02) *
anyway, what i don't really see is why that would influence the archiving community.. i can see how this might be interesting to recording studios, but by the time it reaches the consumer, the format is already decided on. so why would it be relevant to him to save it in as high a sampling rate as possible?
Aha, perhaps I've found the source of confusion. AFAIK the paper assumes archiving to be the process of preventing (historical) analog source material from deterioration by transferring it to a new medium, which in these days usually means a digital format. On the first page of the paper:
QUOTE
Introduction:
The audio archiving community responsible for the preservation of our sonic cultural heritage is interested in adopting a digital conversion and storage format that can be considered transparent by listeners skilled in the art of audio. Therefore, a digital medium having high degree of fidelity to the analog reference is needed.
Analog tapes don't age very well and at some day it will be impossible to play them back at all. If it is important to preserve the audio quality of the original, an as transparent as possible conversion process should be chosen. When the analog originals have become useless, all that's left is the copy, so the choice of archive format is very important. The paper gives some evidence that high sample-rate formats provide a copy that is sonically closer to the original compared to standard rate formats.
Since the cost of a higher rate format in the whole process of archiving is rather small, this could be important information for professional archiving institutions.
Go to the top of the page
+Quote Post
Pio2001
post Dec 25 2007, 23:58
Post #4


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



QUOTE (SoleBastard @ Dec 24 2007, 10:38) *
The whole thing about the lowpassed high resolution version being perceived as better than the non-lowpassed high resolution is very interesting.


QUOTE (boombaard @ Dec 24 2007, 12:02) *
they're saying that if you upsample to the highest SR your sound card can take before playing it back - no matter the quality of the input material - it will be perceived as 'more natural'.


Until a double blind test can show that high resolution is sonically different than standard resolution, any listener can state anything according to his own taste.
All DBT that have succeeded were flawed one way or another.

This one is not flawed, it just failed.

QUOTE (Kees de Visser @ Dec 24 2007, 15:32) *
The paper gives some evidence that high sample-rate formats provide a copy that is sonically closer to the original compared to standard rate formats.


It doesn't. The results are far below statistical confidence. The test failed to show any difference between the stimuli.

The p values are given in figure 1.
They are 0.4 and 0.6, while they should have been at most 0.01 or at least 0.99 for a type I error probability of 5 % (probability to have got the result by chance) because there are four possibilities of success : p1 <0.01 OR p1 > 0.99 OR p2 < 0.01 OR p2 > 0.99, which roughly multiplies the probability of getting a success by 4.

This post has been edited by Pio2001: Dec 26 2007, 00:00
Go to the top of the page
+Quote Post
tarsier
post Jan 1 2008, 06:35
Post #5





Group: Members
Posts: 22
Joined: 26-December 05
Member No.: 26654



QUOTE (Pio2001 @ Dec 25 2007, 15:58) *
This one is not flawed, it just failed.

I'm having a hard time sorting through the math, someone tell me if I'm offbase...

They wanted to test whether sample rates of 44.1 kHz or 8 x 44.1 kHz sounded more like the original source. For those two choices there were two conditions tested: C1 (100 kHz audio bandwidth) and C2 (20 kHz audio bandwidth). (p.3, very last paragraph).

Under C1 conditions, the probability that the high sample rate (8 x 44.1 kHz) sounded more like the analog was 0.4 (p.8 table 3). Does that mean that the probability that the low sample rate (44.1 kHz) sounded more like the analog was 0.6?

And then for C2 (table 3 again), 0.6 probability that high sample rate sounded more like the analog. So does that mean 0.4 probability that low sample rate sounded more like analog?

Am I reading the numbers correctly?

If so, then I have to agree with Pio2001. This test failed to show that either of them sounded "more" like the analog. There was no statistical significance either way. I don't understand how conclusions 2 and 3 on p.9 can say "significantly smaller" and "significantly higher" when it seems like they really should say "a bit smaller" and "a bit higher".
Go to the top of the page
+Quote Post
Kees de Visser
post Mar 10 2012, 10:30
Post #6





Group: Members
Posts: 676
Joined: 22-May 05
From: France
Member No.: 22220



The OP pdf document (2007) has been moved:
http://www.extra.research.philips.com/hera...rs/aar07pu4.pdf

I noticed some extra comments (2009) from the authors:
http://www.extra.research.philips.com/hera...s/aar09pu13.pdf

Kees de Visser
Go to the top of the page
+Quote Post

Posts in this topic


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 16th September 2014 - 02:44