IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
96 vs. 48 or 44.1 kHz sampling --> scientific test, perhaps here is the 1. listening test !
user
post Jan 30 2003, 12:42
Post #1





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



http://www.hfm-detmold.de/texts/de/hfm/eti...ten/seite1.html


http://www.hfm-detmold.de/texts/de/hfm/eti...ten/seite9.html

etc.

(seite = site / page)



it is a summary of German diploma work, written 1998, so it should be independent from companies like Zoony etc.

It was quite a good reading, I hope, somebody of the germans could translate important things, sorry, I have no time.

So, the result was:




44.1 kHz, 16 bit:

they used Tascam-DAT-Recorder DA-30 MKII as reference for CD standard, 44.1/16.

They tell, that this was picked out very clearly from the majority.

(btw, they show a graph, that says, that 16 bit has in mid frequency range a noise level, which is higher than our hearing abilities. very interesting !)

so, 44.1/16 is not sufficient.






48 kHz, 24 bit

one DAC was preferred over analogue original, but why could not be figured out. So there was difference but which ?

another DAC was able to reproduce the analogue original as best DAC of all tested, better than all the 96 kHz DACs !!!!!

in blind test it was 50/50 %, so this DAC could not be distinguihsed from original source.



96 kHz:

Dacs had worse performance than the 48 khZ ones.
but some were "beta-versions...".



Look at the graphs, especially site 9.

Sorry for this short post, but I think, it may interest you...

This post has been edited by user: Jan 30 2003, 12:43


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
Garf
post Jan 30 2003, 12:55
Post #2


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



An dieser Stelle muss aber sicherlich berücksichtigt werden, dass eine unvoreingenommene Bewertung der Klangkriterien kaum möglich war, da sich jeder Testteilnehmer leicht ausrechnen konnte, dass hinter dem verrauschten Signal ein Wandler stecken muss. Die Bewertung, wie sie das Diagramm zeigt, ist daher sicherlich etwas überzeichnet und auf unbewusste, voreingenomme Meinungen zurückzuführen.
Go to the top of the page
+Quote Post
budgie
post Jan 30 2003, 13:23
Post #3





Group: Members
Posts: 341
Joined: 27-November 02
Member No.: 3901



QUOTE
so, 44.1/16 is not sufficient.


HA... laugh.gif HA... smile.gif HA... sad.gif mad.gif (w00t) :x headbang.gif
Very nice, indeed... I should get a Magnum and shoot brain out of my head when I read such "arguments"...
Go to the top of the page
+Quote Post
Garf
post Jan 30 2003, 13:30
Post #4


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



Well, if I understand the German right, they did illustrate that listeners could distuinguish a CD quality recording from an 'original' (how did they play back _that_ ?).

The thing is that the results seem to depend too much on the used DAC rather than on the format itself, e.g. the 48khz DAC being chosen over the 96khz ones.
Go to the top of the page
+Quote Post
user
post Jan 30 2003, 13:49
Post #5





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



just a moment, it doesn't help, if you cite single parts of texts.

btw, those results are quite agreeing to main opinions of this board, there is no reason to bash that work.


@garfs posted lines:

they refer to the first test part, where i am not so clear about, if they did it blind.

well, they tell, the noise of that 44.1/16 source (DAT) was easy to listen, so they ranked 16/44.1 as worst.

btw, in the latter test, which is blind, 16/44.1 is rated worst, too, very clearly.




Well, somewhere they write clearly, that this test showed more the single characteristics of each DAC.

Compare the 24/48 DACs and the 24/96 DACs.

(at that time, 1998 !!!!, the 96/24 DACs were not perfect, not as good as the 48/24 DACs. This is one result of their test.

They write, that obviously the sound is more infleunced by the analogue in and outs circuits of the DACs, then by the digital circuits....

Well, and as they took only one example for 16/44.1, a Tascam DAT (which is used in a lot studios, the reason, why they took it, so, it cannot be so bad ?) it can be, that this DAT has perhaps a worser analogue output than other 44.1/16 DACs, who knows ?!


So, they have not only the result, that their Tascam DAT produced some noise at 44.1/16, they write as result, that very probably 96 kHz sampling is overkill.
The good blind-tested 48 kHz DACs proved, that transparency is already available at 48 kHz sampling (which does not exclude, that 44.1 kHz sampling , 20 kHz sound, would be transparent, too.
But these 48 DACs had 24 bit.

So, they tell, that the 24 bit is an improvement over 16 bits.
Not the 96 kHz are an listenable (ABXable) improvement, 96 kHz are waste of space, which we will agree mainly, without great headaches.


well, this 24 bit better than 16 bit thing is unfortuanetly not really tested by them. Their test-setup was mainly for 96 kHz against other sampling frequencies.
And therefor they have a nice result, 48 kHz perfect.


Their graphs are unfortunately not the best ones.
you can get some valuable information out of them.
but in this summary they give only very little numbers, statistics.
I assume, that there is more statistical proof in the real diploma work.

But the whole test-setup, and how it is written, shows, that they wanted to get real blind-tested results, like we want here at HA.


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
NumLOCK
post Jan 30 2003, 13:55
Post #6


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



Garf, I have not read this article yet but it seems to me, that there might be an explanation for the 24-bit difference that we haven't thought of. A few weeks ago this problem just popped up in my mind.

Here it is: the Nyquist theorem is true if - and only if - each sample is stored with infinite precision.

In other words: if we truncate each sample of the original signal to 16 bits, the added quantization noise will - after the signal passes through the DAC - affect not only the noise floor of the resulting signal, but also various issues such as phase, separation between frequencies, etc.

I've been thinking about verifying that in Matlab, but had no time for that so far. Does it make any sense to you ?

[Edit]
Now about the test: It seems pretty stupid to me. I mean, why not use the SAME 24-bit/96kHz DAC each time, and downsample/requantize the signal for the lower sampling rate tests ? Sounds like the only way to avoid comparing the DAC's rather than the sound quality.
[/Edit]

This post has been edited by NumLOCK: Jan 30 2003, 13:59


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post
user
post Jan 30 2003, 13:56
Post #7





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



QUOTE (Garf @ Jan 30 2003 - 01:30 PM)
Well, if I understand the German right, they did illustrate that listeners could distuinguish a CD quality recording from an 'original' (how did they play back _that_ ?).

The thing is that the results seem to depend too much on the used DAC rather than on the format itself, e.g. the 48khz DAC being chosen over the 96khz ones.

yep, they write it, that every DAC will have its own influence due to analogue in and out circuits of each DAC.


actually they did NOT take a good CD-recording.

They played music in another room and used best available studio techniques to (record and) transmit it to the several to be tested DACs.



They had one very important test-setup for testing the main question, they asked: 96 vs. 48 kHz.
as this would be dependent on DAcs, too, they took a 96 kHz recording, and used some downsampling to 48 kHz, and then playback through 96 kHz, again, to test this 48 kHz-signals gainst the 96 kHz-originals , both playable through same 96-DAC.


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
NumLOCK
post Jan 30 2003, 13:58
Post #8


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



QUOTE
They had one very important test-setup for testing the main question, they asked: 96 vs. 48 kHz.
as this would be dependent on DAcs, too, they took a 96 kHz recording, and used some downsampling to 48 kHz, and then playback through 96 kHz, again, to test this 48 kHz-signals gainst the 96 kHz-originals , both playable through same 96-DAC.

Which, unsurprisingly, resulted/results in virtually no perceivable difference for the listeners.


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post
F1Sushi
post Jan 30 2003, 15:00
Post #9





Group: Members
Posts: 158
Joined: 30-August 02
Member No.: 3236



I think it's pretty clear from this "study" that the participants were evaluating the anti-aliasing filters and not the signal converters. This issue comes up a lot in these tests, and this is where most of them fall flat.

Not that there's anything wrong with evaluating analog domain front ends, but it shows how well the "studiers" understand the nature of the beast, and the need for a proper setup. I seem to recall David R working with what appeared to be a proper setup for this experiment way back (something about double blind tests at a tradeshow comes to mind)...
Go to the top of the page
+Quote Post
ChristianHJW
post Jan 30 2003, 16:33
Post #10


Matroska developer


Group: Members
Posts: 922
Joined: 29-September 01
Member No.: 74



Here is how i would make such a test :

1. Record a jazz combo live with a simple, classic 2 microphone setup ( to preserve runtime delay best possible ), using a very high quality 24 Bit/ 96 Khz ADC

2. Downsample the recorded digital signal to 44.1 and 48 Khz afterwards on a PC, with a normal FFIR filter ( or whatever ), but using a very high internal precision to avoid rounding errors ( like 32 bit FP ), and dither the signal

3. Play back those downsampled signals using a state-of-the-art 24 Bit / 96 KHz DAC and an excellent stereo monitoring system

4. Let the user in an ABX decide what sounds 'best' or most natural to them

If you use different DACs you will hear the differences between them, and not the sampling frequency.


--------------------
Support matroska - the bestest vapourware project ! http://www.matroska.org
Go to the top of the page
+Quote Post
budgie
post Jan 30 2003, 16:54
Post #11





Group: Members
Posts: 341
Joined: 27-November 02
Member No.: 3901



Hey, hey, people... stop, please!!! Sometimes I just wonder like Alice in the Wonderland rolleyes.gif what is being discussed here... Music stays completely aside, the talk si about something what almost nobody really needs. Just remember the old days back in 1955 to 1965 and excellent jazz recordings especially from Rudy van Gelder era done just with two mics in his living room on 2- (or 3-) track machines... The songs being recorded in one take, maybe some takes were made for each song and the whole album was made within a days (very often in one or two...) The recordings (be it on LP or later a CD re-issue) have excellent quality, maybe with higher noise but who really concerns? This is what really matters, real music in the real world... not these academical extracts... laugh.gif
Go to the top of the page
+Quote Post
F1Sushi
post Jan 30 2003, 17:25
Post #12





Group: Members
Posts: 158
Joined: 30-August 02
Member No.: 3236



You're certainly entitled to your opinion (so why do you hang out at hydrogen audio, then?). Back to the issue...

unsure.gif
Go to the top of the page
+Quote Post
Guest_SK1_*
post Jan 30 2003, 17:40
Post #13





Guests






This "study" is completely flawed.
Untill someone does a test like NumLOCK and ChristianHJW suggested, i have nothing to learn from that "study".
Go to the top of the page
+Quote Post
user
post Jan 30 2003, 20:00
Post #14





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



Sorry SK1,

please tell me, what is flawed in this study ?

Can you read german ?
Who translated for you ?


Tell facts, what the flaws are !


Hey, this study needs at least 9 sites. And these 9 or 10 sites are only an extract out of perhaps 60 sites, dunno.



cite:

" Here is how i would make such a test :

1. Record a jazz combo live with a simple, classic 2 microphone setup ( to preserve runtime delay best possible ), using a very high quality 24 Bit/ 96 Khz ADC

remark user: afaik I have read, this has been done nearly that way. They had some musicians playing, perhaps 1, 2 or 3, classical instruments, and this was recorded at the best way, I remember that classic 2-microphone setup.

Well, you should all know, that this "High-School (a kind of special university, specialzed on studies for becoming sound-ingeneer)", where the test took place, is the only kind besides another in Berlin, in Germany. So, they have the knowledge, all (if not more), like we have here at HA.
They have around 40 - 50 students, all becoming sound-ingeneers at the end.
btw, these guys were the test-listeners.
So they would have had enough to perform a solid statistic, I assume (well, the information about carried out statistics is very small in this article, but there is some. I want to add, that 2 main lessons of the students are maths and physics, specialized on sound....) The statistical stuff will fill the a lot of sites of the official diploma work.............





2. Downsample the recorded digital signal to 44.1 and 48 Khz afterwards on a PC, with a normal FFIR filter ( or whatever ), but using a very high internal precision to avoid rounding errors ( like 32 bit FP ), and dither the signal


reamrk user: in one test-setup, this was carried out similarily to 48 kHz, I had mentioned it above. They were aware of the fact, that every DAC may sound a different way...




3. Play back those downsampled signals using a state-of-the-art 24 Bit / 96 KHz DAC and an excellent stereo monitoring system


remark user : done, written above.




4. Let the user in an ABX decide what sounds 'best' or most natural to them

If you use different DACs you will hear the differences between them, and not the sampling frequency. "

remark user:
that was carried out, otherwise, I wouldn't have wasted my time, to inform you about this study.

Well, I haven't found a flaw in this study. They were aware of all the mentioned things until now.
Why are some of you critizising this study ?
If you find a flaw, describe it. But you should not tell here possible flaws, which *might* have occured, like if they were beginners.


hmm, I know, it is difficult for most of you, to read those german sites.
(I assume, that this study wasn't spreaded so far, as it tells quite clearly, that 96 kHz is not necessary, 48 kHz are enough. The big music-industry had surely no interest to THIS result, lol !!!)
About that 16 bit vs. 24 bit thing, the study tells nothing new, perhaps I have written/interpreted too much in my first post.)
I have read, what guys these 40-50 students,( = participants of the blind -tests ) are.
They are selected out of a quite big number of volunteers, who want all to become sound-engineer, master for recording professional in studios or live etc.
So, they select the best ones, theoretical and practical skills.
Then they let them study at thier school.
So, the guys are quite young (20-25 max.) (haven't lost thier hearing abilities so far...)
and they are trained for musicality listening. They must play instruments , before they start their studies there...
and knowledge about electronics, acoustics, electrical, physics are reqwuired too, ot they are teached at that school.

In fact, they would be ideal HA-members, if they would start dealing with lossy codecs, too, and some training to listen to artifacts of lossy encoders, but this could be learned quite fast...

This post has been edited by user: Jan 30 2003, 20:01


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
F1Sushi
post Jan 30 2003, 21:23
Post #15





Group: Members
Posts: 158
Joined: 30-August 02
Member No.: 3236



QUOTE (user @ Jan 30 2003 - 03:00 PM)
"in one test-setup, this was carried out similarily to 48 kHz, I had mentioned it above. They were aware of the fact, that every DAC may sound a different way..."

Can you be more specific as to how this was tested? What do you mean by "this was carried out similarily to 48 kHz" ? Your first post didn't suggest this test approach was used...

Isn't the fact that this study was conducted 5 years ago somewhat important? Multi-bit sigma-delta modulators weren't around then.
Go to the top of the page
+Quote Post
NumLOCK
post Jan 30 2003, 21:26
Post #16


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



QUOTE (user @ Jan 30 2003 - 08:00 PM)
[...]
what is flawed in this study ?
[...]

A different DAC was used each time, which is not the safest thing to do when you're desperately trying to notice differences beyond 90dB SNR !

Remember, what you basically want, is to distinguish between the following two binary 24-bit samples, just by listening:

YYYYYYYYYYYYYYYY00000000 (24 bits truncated to 16 bits)
YYYYYYYYYYYYYYYYXXXXXXXX (24 bits not truncated)

If any of both converters has a linearity problem below 2^-15, it's already too much for the test to be valid.
The main reason for this, is that I doubt seriously that a 16-bit converter would produce perfectly bit-accurate output down to the LSB.


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post
user
post Jan 30 2003, 22:11
Post #17





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



Numlock,
now I am even asking myself, if you have read the study ?

they have never tested for 24 bit to 24->16 truncated, or 16 bit.

hey, they tested 96 vs. 48 kHz @ 24 bit,
those single 44.1/16 test showed the bottom line...



hmm, no, they have not tested every time another DAC.
That was on purpose.

If they would have done the opposite:
only one DAC from one company, then we would cry (and we were right then): silly, test based only on one DAC, product.

They compared each DAC vs. the original.
and the testers should tell, which sounds better, DAC or original.
and that leaded to intersting varying results.

It showed, that DACs behaved quite different !

But it showed for the one 48 kHz DAC, that this was a perfect DAC, its output was indistinguishable from source/orignal. And the other 48 kHz DACs showed excellent performance, too.

For the new 96 kHz DACs the results were different.

This post has been edited by user: Jan 30 2003, 22:18


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
jesseg
post Jan 30 2003, 22:57
Post #18





Group: Banned
Posts: 218
Joined: 22-December 02
Member No.: 4194



QUOTE (NumLOCK @ Jan 30 2003 - 02:26 PM)
...I doubt seriously that a 16-bit converter would produce perfectly bit-accurate output down to the LSB.

What's the LSB? Im working on 3wk and were looking at a pair of these... http://www.apogeedigital.com/products/prod_minime.html
If I am able to get them at my house before hand to configure Ill do some tests, and record the results to wavs and flac them so you can abx yourself... If 3wk does that. They might decide to stay with an all sdpif setup, instead of integrating this one "goodie" we wanna get. dry.gif

anywho, whats LSB?
Go to the top of the page
+Quote Post
ChristianHJW
post Jan 30 2003, 23:25
Post #19


Matroska developer


Group: Members
Posts: 922
Joined: 29-September 01
Member No.: 74



I took the time and read the complete article.

user is correct, those people made a good job ( if only they didnt use the crappy Manger speakers biggrin.gif ), the test was very well structured and the results seem clear.

The only thing i missed is what i was stating above, that is to compare the 96 KHz sampled signal with the very same signal, but limited in badnwidth. They have not done so, but always compared the sampled signal with the analog reference ( live band ).


--------------------
Support matroska - the bestest vapourware project ! http://www.matroska.org
Go to the top of the page
+Quote Post
Doctor
post Jan 30 2003, 23:37
Post #20





Group: Members
Posts: 160
Joined: 16-January 03
Member No.: 4597



Least Significant Bit in this case.
Go to the top of the page
+Quote Post
jesseg
post Jan 30 2003, 23:39
Post #21





Group: Banned
Posts: 218
Joined: 22-December 02
Member No.: 4194



thanks, ill read up about its important, if something had a high lsb, that would be worse right?
Go to the top of the page
+Quote Post
AgentMil
post Jan 31 2003, 07:21
Post #22





Group: Members (Donating)
Posts: 584
Joined: 19-December 01
From: Australia
Member No.: 688



Slightly OT biggrin.gif

Sound is so subjective, in that I think most test are always flawed to some extent, but if the general consensus is for a certain encoder/amplifier/decoder/DAC to be better than another then the test can be said to have verifiable results.

What is your take on testing sound quality? Is it completely subjective or objective? I would like to hear other thoughts on this topic.

Cheers


--------------------
-=MusePack... Living Audio Compression=-

Honda - The Power of Dreams
Go to the top of the page
+Quote Post
NumLOCK
post Jan 31 2003, 09:08
Post #23


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



QUOTE (user @ Jan 30 2003 - 10:11 PM)
Numlock,
now I am even asking myself, if you have read the study ?

I tried, yes.

QUOTE
they have never tested for 24 bit to 24->16 truncated, or 16 bit.

hey, they tested 96 vs. 48 kHz @ 24 bit,
those single 44.1/16 test showed the bottom line...

Sorry, I really thought that besides the 96kHz-DAC vs 48kHz-DAC test (where 48 was better), they tried to use the dCS 952 (Seite 7) in 16, 20 and 24-bit modes and compare. Well, it turns out my German definitely sucks. Indeed they were just mentioning the DACs' specs. My apologies.

QUOTE
hmm, no, they have not tested every time another DAC.
That was on purpose.

If they would have done the opposite:
only one DAC from one company, then we would cry (and we were right then): silly, test based only on one DAC, product.

They compared each DAC vs. the original.
and the testers should tell, which sounds better, DAC or original.
and that leaded to intersting varying results.

Ok.

QUOTE
It showed, that DACs behaved quite different !

But it showed for the one 48 kHz DAC, that this was a perfect DAC, its output was indistinguishable from source/orignal. And the other 48 kHz DACs showed excellent performance, too.

For the new 96 kHz DACs the results were different.

But, why didn't they take three 96-kHz DAC's, and compare what they produce when the input changes ? From what you said, using different DAC for different frequencies really is biased by the component's properties, isn't it ?

Well, to my mind, here is how I understood the tests :
- is a 48kHz converter ALWAYS perceptually as good as a 96kHz one ? => this test concludes "yes".
- is a 24kHz lowpassed signal perceptually as good as the analog one ? => also, result is "yes".

Missing stuff would be :
- is 20-bit and more, better than 16 bits ? if it is, then at what sampling rate does the difference vanish ? => would be a nice test to do.
- does the reference analog source REALLY contain frequencies above 24kHz ?

Again, sorry about by misunderstanding of that test.
Btw, I definitely agree with your remark about the music industry.

This post has been edited by NumLOCK: Jan 31 2003, 09:49


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post
budgie
post Jan 31 2003, 09:16
Post #24





Group: Members
Posts: 341
Joined: 27-November 02
Member No.: 3901



QUOTE
...it tells quite clearly, that 96 kHz is not necessary, 48 kHz are enough...


NumLOCK:

Do you remember what I wrote you in the PM last week? 20bit/48kHz is sufficient enough for any high(est) end audio... rolleyes.gif
Go to the top of the page
+Quote Post
NumLOCK
post Jan 31 2003, 09:23
Post #25


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



QUOTE (budgie @ Jan 31 2003 - 09:16 AM)
NumLOCK:

Do you remember what I wrote you in the PM last week? 20bit/48kHz is sufficient enough for any high(est) end audio...  rolleyes.gif

Well, yes I know, I know.. biggrin.gif

I won't argue with the 48kHz, since I agree with it wink.gif

To be sincere I'm still wondering what implications a truncation to 20bit really has. Doesn't the Shannon only apply to discrete, ANALOG domain ? Does truncating to N-bits only rise the noise floor ?

Cheers


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st August 2014 - 03:24