Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 24/96 digitalization - can it be audible (Read 36928 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

24/96 digitalization - can it be audible

Reply #26
I don't think that we have to speculate about possible minor shortcomings of the Tascam when the test is known to have two major problems:

- Level matching within 0.2dB is insufficient and could be well audible
- Clipping

It should be repeated with levels matched within 0.1dB and lowered input level to give really significant results.

24/96 digitalization - can it be audible

Reply #27
I'm no statistics expert but it seems like too little data. Have you tried flipping a coin 130 times instead and processing the results the same way? What kind of conclusion would you jump to if it gave you the same result?

24/96 digitalization - can it be audible

Reply #28
Just to be clear, this is all about an average score?  All the individual scores failed to reach the statistical significance threshold?
Yes - but keep in mind that due to low number of trials and due to correction for multiple participants, the threshold for individual analysis was very high - 12/13 =92.3%. Unfortunately we could not run more trials, with 50 trials and 10 participants just 74% (37/50) would suffice.

82/130.

Assuming you didn't cherry pick the data, I reckon it's fair to take that as one block.
I did not cherrypick. All data from all participants are used, exactly as reported in the asnwer cards. Actually, one of the top-scorer's errors was a correction from initially correct response, had he not corrected, he would pass the individual statistical threshold. But we take his (wrong) correction as final.

I refrained from taking the data as one block as you suggested, because they were 13 submeasurents (trials) in 10 measurements (participants), and assuming individual differences in perceptual abilities, the 13 submeasurements within each of 10 participants were not independent. Anyway, the 95% confidence interval of the 82/130 proportion is 0.5452 to 0.7089 (or 0.5413 to 0.7125, using another method), it does not include 0.5, so would I consider the proportion to be significantly different from 0.5.

I'd be very worried about a 0.2dB level difference, and quite worried about clipping (depending on the content).
Yeah, clipping is not good, and we should have avoided it. As i said though, there was very little of it, just a few cut of peaks over tens of seconds of recording.

As of the level difference, are there any published data showing sensitivity to level differences <0.2 dB in complex/musical signals? Anyway, if this is significant, we would need a different A/D/A device, one that it is closer to ideal unity gain

But the multiple listeners issue is bothersome.  Is there any possibility that the listeners influenced each other?
This cannot be excluded, but there was no way to get access to the room and equipment for 10 individual sessions. We have no reports of such influence from the participants, but subconscious influence is theoretically possible

Also, as far as applicability, we don't tend to listen to our hi-fi that way, with all that acoustic interference.
What kind of acoustic interference are you referring to?

- Level matching within 0.2dB is insufficient and could be well audible
[...]
It should be repeated with levels matched within 0.1dB and lowered input level to give really significant results.
Again, I'd appreciate references that established jnd for intensity in complex signals to be <0.2 dB

I'm no statistics expert but it seems like too little data. Have you tried flipping a coin 130 times instead and processing the results the same way? What kind of conclusion would you jump to if it gave you the same result?
I believe this part has been taken care of properly without physically flipping a coing. That's what we have math for.
Ceterum censeo, there should be an "%is_stop_after_current%".

24/96 digitalization - can it be audible

Reply #29
I'd be very worried about a 0.2dB level difference, and quite worried about clipping (depending on the content).
Yeah, clipping is not good, and we should have avoided it. As i said though, there was very little of it, just a few cut of peaks over tens of seconds of recording.

As of the level difference, are there any published data showing sensitivity to level differences <0.2 dB in complex/musical signals?

- Level matching within 0.2dB is insufficient and could be well audible
[...]
It should be repeated with levels matched within 0.1dB and lowered input level to give really significant results.
Again, I'd appreciate references that established jnd for intensity in complex signals to be <0.2 dB
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.

* - The closest is 0.25dB under quite different conditions: F.E. Toole and S. Olive, "The Modification of Timbre by Resonances: Perception and Measurements", JAES vol 36, # 3, March 1988, pp 122-142

Cheers,
David.

24/96 digitalization - can it be audible

Reply #30
I believe this part has been taken care of properly without physically flipping a coing. That's what we have math for.

Well, any statistical analysis is easily misinterpreted or skewed towards a certain point by formulating the tested hypothesis in a certain way.
Why not test a hypothesis that all of the participants were just guessing and their scores were drawn from a coin-flip-distribution random number "generator"?
That was my point, sorry for not making it more apparent.

24/96 digitalization - can it be audible

Reply #31
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.


Sure. I am trying to find out how close level-matching needs to be. In other words, what is the basis and the limit for the "better matching" requests. I mmagined that we would do a similar test with levels matched to 0.06 dB, and how do I know that a HA'er won't say "hey, it should have been 0.05 or better".


Why not test a hypothesis that all of the participants were just guessing and their scores were drawn from a coin-flip-distribution random number "generator"?
That was my point, sorry for not making it more apparent.


I suggest that you re-read the thread, paying special attention to "binomial test", "Šidák correction", and "confidence interval of proportion".

Ceterum censeo, there should be an "%is_stop_after_current%".

24/96 digitalization - can it be audible

Reply #32
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.

Sure. I am trying to find out how close level-matching needs to be. In other words, what is the basis and the limit for the "better matching" requests. I mmagined that we would do a similar test with levels matched to 0.06 dB, and how do I know that a HA'er won't say "hey, it should have been 0.05 or better".

AFAIK the common standard for level-matching is <= 0.1dB. I don't know whether this value has been empirically deducted but it seems to be sufficient.

24/96 digitalization - can it be audible

Reply #33
FWIW, I recall once being able to successfully ABX a synthetic signal to at least a 0.05db difference in volume, with IEMs. But I did not think I could have been able to repeat that feat with speakers or with "real" music, and I've always worried that those Etys might have distortion levels varying significantly with sound level....

24/96 digitalization - can it be audible

Reply #34
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.


Sure. I am trying to find out how close level-matching needs to be. In other words, what is the basis and the limit for the "better matching" requests. I mmagined that we would do a similar test with levels matched to 0.06 dB, and how do I know that a HA'er won't say "hey, it should have been 0.05 or better".

Good point.

I guess one answer woul be that "the more controversial your findings are, and the less able others are to repeat your test, the more painstaking you have to be in controlling these things in order to make a difference to 95% of the sane readers. The last 5% will dispute your findings no matter what you do".

0.1dB is a number that I have seen several times. It might be a number that is both practical to achieve and well enough ahead of what we believe that humans can distinguish.

I think that at this point, getting independent confirmation is most important.

-k

24/96 digitalization - can it be audible

Reply #35
FWIW, I just ABXed a song with a 0.1 dB difference (normalized the original, which was at 0.0 to -0.1). I used headphones and focused on a very short sample repeatedly. So probably this is not the same methodology as you used, but it's still something I'd look into for such tests.

24/96 digitalization - can it be audible

Reply #36
FWIW, I just ABXed a song with a 0.1 dB difference (normalized the original, which was at 0.0 to -0.1). I used headphones and focused on a very short sample repeatedly. So probably this is not the same methodology as you used, but it's still something I'd look into for such tests.

An impressive result. The only drawback is if the original was 0.0 it could possibly have caused some minor clipping. It would have been better if you had normalized to, say, -1,0 and -1.1 instead, just to be really sure that there's no clipping.

24/96 digitalization - can it be audible

Reply #37
An impressive result.
You can try for yourself with this online hearing test:
http://www.audiocheck.net/blindtests_index.php
Since it's relatively easy to match levels in the OP setup, 0.1 dB seems reasonable, just to eliminate yet another variable.
OTOH moving one's head will result in larger level (and spectral) differences than 0.1dB, so perhaps it's a moot point.


 

24/96 digitalization - can it be audible

Reply #39
An impressive result.
You can try for yourself with this online hearing test:
http://www.audiocheck.net/blindtests_index.php


Using headphones, and coming home from work after a ride in the bus and being very active in the kitchen, I dropped to 6/10 at 1dB. A second pass with more focus got me back to 10/10.
0.5dB was a guessing game. For completeness I tried 0.2 and 0.1, but I heard nothing. Pitch test got me to 10c, and then it was all the same.

The 440Hz tone made my ears bleed after a while, though.

24/96 digitalization - can it be audible

Reply #40
FWIW, I just ABXed a song with a 0.1 dB difference (normalized the original, which was at 0.0 to -0.1). I used headphones and focused on a very short sample repeatedly. So probably this is not the same methodology as you used, but it's still something I'd look into for such tests.

An impressive result. The only drawback is if the original was 0.0 it could possibly have caused some minor clipping. It would have been better if you had normalized to, say, -1,0 and -1.1 instead, just to be really sure that there's no clipping.

I was thinking the same, actually. And also since one file was processed and the other wasn't. So I did as you suggested, one at -1.0 and the other at -1.1 (separate processing, of course). I managed a 8/8, then perhaps got a bit impatient, stopped at 9/11. It's not the 10/10 I'm usually aiming at, but take it as you will. I might give it another shot on a calmer day. (It's stormy right now.)

24/96 digitalization - can it be audible

Reply #41
Just a question. Can you guys replicate this kind of accuracy at 0.1-0.2 dB difference if you take off your headphones and go to another room for a few minutes between trials? This is how our test was conducted (and we used loudspeakers and music, not headphones and 440 Hz tone).
Ceterum censeo, there should be an "%is_stop_after_current%".

24/96 digitalization - can it be audible

Reply #42
I haven't tried yet, so I won't make any claims. But if I were to guess I'd say no. Even with careful listening on the headphones it wasn't easy and as I've hinted before I had to constantly repeat the involved samples to spot the differences.

But since we're scrutinizing this, what's the reasoning for only using one song for the test and this song in particular ("Falling Alice")?

24/96 digitalization - can it be audible

Reply #43
But since we're scrutinizing this, what's the reasoning for only using one song for the test and this song in particular ("Falling Alice")?


Only one song - mostly time constraints. We had 13 trials, add time needed for device switching, and you will realize that using multiple songs was not really feasible.

Why this song? Well, we had a pretty long discussion with very diverse suggestions, and this song was suggested by one participant mainly because of variety of timbres involved, including human voice, and relatively transparent texture. And everybody agreed to use it :-). An additional factor was that a fully analog (i.e., old enough) LP in good condition could be provided.

Most people agreed that the genre/style of music was not too unfamiliar for too many of them. Anecdotally, familiarity turned out to be a factor, as the top-scorer was the owner of the LP....
Ceterum censeo, there should be an "%is_stop_after_current%".

24/96 digitalization - can it be audible

Reply #44
I previously kinda implied that using headphones made the ABX easier.
I've now finally tried the 0.1 dB ABX with speakers and I thought it was actually easier than with headphones. Uploaded files here.

I think usually for lossless-lossy ABXing headphones are better, but I guess with volume differences speakers could have an advantage. (Feeling the sound with the whole body etc.)


I still didn't try ABXing the same way you did for the test, though. I was repeating the short pieces back and forth. But theoretically, if the difference was in fact 0.2 dB instead of 0.1 dB, I would imagine it could be possible even with just a single listen and a bigger pause in between.

24/96 digitalization - can it be audible

Reply #45
:F

If it's easier with speakers, then I'm inclined to at least suspect that the difference might be distortion-induced.

24/96 digitalization - can it be audible

Reply #46
:F

If it's easier with speakers, then I'm inclined to at least suspect that the difference might be distortion-induced.


That's the second post you've made in this thread suggesting that a small difference in volume (the other time, .05 dB) was detectable due to distortion.

By what mechanism would distortion differ significantly over a .05dB power change?  Or even 1 dB? 

I can certainly accept that speakers can have audible distortion.  The question is how much the distortion can change, beyond just scaling with the main signal, over a very small range of level.

24/96 digitalization - can it be audible

Reply #47
I have a couple of queries regarding methodology (and I've only read your brief description in English so I don't know the details)

First, if there's a difference in delay, it's possible that speaker-to-stylus feedback (e.g. vibrations) could influence the sound quality in different ways. I guess high speed reel-to-reel magnetic studio tape would be an analogue alternative source that is immune to this and could be recorded (once) from the original vinyl source or real instruments.

During listening, the test organizer remained in the back of the room, invisible to the listeners.

Invisibility is certainly important, as is truly random order, so that's good procedure. It might be important that there is no other possible communication. For example, use a lamp rather than speaking to indicate the system is ready for testing and maintain a constant time-delay between auditions regardless of whether the digital component is switched into or out of use or left as the previous test (and constant time the light is lit).

A further concern with communal listening is that it's possible for some of the listeners to be influenced by each other (even subconsciously), causing their answers to be correlated rather than completely independent which can mess up the statistics and cause significance to be incorrectly inferred on a flawed assumption of statistical independence.

It's amazing how subtle things can be picked up subconsciously without trying. For example I recently heard a story of a man inadvertently made his wife think she was psychic. He was doing a really awful card trick with a friend when his wife came in (all three were staunch rationalists). The husband said "OK, guess the colour of the next card" and the friend would say "Black". Then "Right, what colour is this one?" and friend replied "Red.". They got it wrong now and then to make it seem plausible and the wife asked to have a go. Bizarrely, and unexpectedly to all concerned, she started to guess perfectly and was freaked out and thought she could be psychic or getting extra-sensory perception. She was livid with her husband when he explained what had happened. It turns out she'd subconsciously picked up the secret code without consciously realising it, and in this case it was right for red, okay for black. One might presume that the brain is reinforcing neural pathways that link various word to the same outcome and while it feels like you're picking red or black out of thin air, the tenuous neural connection can become strong enough to tip the balance of near-randomness in favour of the right answer with remarkable consistency.

It's possible that the person who switched the signal path might tap their foot subconsciously behind the curtain in a different way despite trying desperately not to influence the listeners at all.

This sounds like a good experiment, and it might be a significant result, but it can be so difficult to eliminate all possible sources of statistical correlation and effects beyond those intended for measurement. If you do test again, best of luck. It might even be interesting to rewire the Tascam internally to disconnect the line out and wire directly from the line in (with a plain analogue cable) while leaving the line in attached to the ADC if the clipping or level lights can be observed to provide a null control trial unbeknown to all participants.
Dynamic – the artist formerly known as DickD

24/96 digitalization - can it be audible

Reply #48
There were 10 listeners (neither the test organizer nor me participated; he was switching the connections, I was several thousand kilometers away). All listeners listened together, being in the same room. They left the room for connection switching. During listening, the test organizer remained in the back of the room, invisible to the listeners.


The presence of a person who knew the actual status of each trial in the room is to me the most obvious weak spot in the test.

It is unecessary for him to be there, and it should have never happened.

I can hypothesize a long list of tells that may have unconsciously communicated what he knew to the people in the room. Of course I wan't there and don't know what actually happened.

24/96 digitalization - can it be audible

Reply #49
Is there a possibility that some convertor chips are designed from scratch to convert at multiples of 44.1 but have the ability to resample to multiples of 48 and vice versa? Are these things supposed to be equally good at conversion (as far as the human ear is capable) at all sample rates?