IPB

Welcome Guest ( Log In | Register )

The Absolute Sound FLAC article
SoAnIs
post Jan 7 2012, 04:19
Post #1





Group: Members
Posts: 9
Joined: 28-August 07
Member No.: 46568



My father subscribes to "The Absolute Sound," and in the latest issue they have an article about FLAC. Of course they have no (or very minimal) double-blind testing (something both my father and I believe is necessary for any review to be valid) and a history of endorsing bullshit, so we take their results with a truckload of salt.
They score their comparisons on a scale from 0-200 points, with CD audio as 100 points. The scores are generated by A/B comparison (not A/B/X) to a variety of reference qualities. The first issue crops up with this. 0 = Kazoo. 50 = 320kbps MP3. 100 = Red Book CD, 44.1khz/16 bit. 110 = 48khz/24 bit upconverted CD. 120 = 88/24 upconverted. 130 = 96/24 upconverted. 150 = 176/24 upconverted. 160 = 192/24 upconverted. 170 = 192/32 upconverted. 180 = SACD/HRx/High-res downloads. 200 = "Master Tape?". There are no references between 50 and 100. Furthermore, a "very large" difference of 50 points would "require only one A/B comparison and is so large that a 24-hour hiatus between listening sessions would still elicit the typical audiophile expression of 'Holy Cow! This is a night and day difference!'"
Both their reference systems used on-board sound cards capable of 192khz S/PDIF output into a PS Audio PWD DAC. Motherboard brand and model is not disclosed. Actual sound-card model is not disclosed.
This seems quite suspicious to me. I have been unable to successfully detect differences between 320kbps CBR MP3, wave ripped from CD using Exact Audio Copy, and CD Audio using Foobar2000's ABX comparator plugin. I used Epica's "Design Your Universe" tracks "White Waters" and "Burn to a Cinder" as my references this time. I used the onboard audio of my motherboard (an ASUS P6X58D Premium, Realtek ALC889 audio, with 24-bit/192khz default output set in Windows 7) and a set of Grado SR-60 stereo headphones. My CD drive is a Lite-On iHAS424. I don't have a fancy $4000 DAC. My ABX test results were no better than guessing, nowhere near a "night and day" difference audible with a 24-hour gap. I sincerely doubt that the lower quality of my headphones and DAC would be enough to mask such a difference. Thus, I doubt their measurement scale's accuracy.
They did their playback comparisons using JRMC (proprietary) with 192/32 realtime upsampling. They never once mention using the reference FLAC encoder/decoder, or even FLAC frontend.
The top 3 programs in sound quality were:
JRMC: 150 score for HRx 176.1khz/24-bit to FLAC
dBPowerAmp: 145
Foobar2000: 140
Original HRx wave file: 180
So they're claiming a large drop in sound quality.
Later they claim that converting from FLAC to WAV with Foobar can increase sound quality (where would the extra data be coming from?) and that other programs decrease sound quality. In all cases they claim a decrease in sound quality when compared to the original CD.
They also (via listening tests, no md5 hashing or such) claim that Wav->FLAC->Wav conversions create cumulative degradation. This is, of course, a very big claim.
So I took a wav file (a rip of "White Waters", mentioned above) and used the FLAC reference encoder to compress it. Then to decompress. I compared the MD5 hashes of the two .wav files, and they were the same. Just in case some strange miracle had occurred and I'd found an MD5 collision I compared the SHA-512 sums, and those were also the same. The two files were bit-for-bit identical. Just for fun, I repeated the wav->FLAC->wav process 10 times, and the files still had identical hashes. I then ABX tested the original and final files, and was unable to do better than guessing.

TL;DR: Bullshit detected.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
milosz
post Jan 18 2012, 20:47
Post #2





Group: Members
Posts: 1
Joined: 18-January 12
Member No.: 96535



I agree that two bit-identical files should sound the same.

However playing back two bit-identical files side-by-side or sequentially might engender some DAC or other digital-domain data handling difference. Just because the files on your drive are bit-identical doesn't mean your DAC is going to play them back exactly the same. The DAC should play them back exactly the same, but I think there's a possibility it might not. Things like jitter, LSB noise, etc can all come into play. Just a guess, but I would think that EVERY TIME you play a file back- whether from CD or hard drive or whatever- it will decode to analog a few parts per million (billion?) differently.

What I would like to see is a bit comparison in a pair of buffers to see if two bit-identical files present exactly the same bitstreams to the DAC. Does playing a file back happen without error? It might, I don't know. Does digital=>analog conversion happen exactly the same for each set of data, down to the last microvolt? I don't know.

But this is about file playback in general, not about some kind of file conversion loss or file-type-specific effect.

THAT said, I don't see that you could draw a conclusion that FLAC sounds worse than WAV given the experimental setup used by the authors of the Absolute Sound article. They could POSSIBLY conclude that the way their setup played these files back yielded some kind of differences- but you could not extrapolate this to say that FLAC just sounds worse than WAV in general. They failed to control for all the variables, by a wide margin. Very poor experimental design, and inexcusable hubris on the author's part in drawing their conclusions based on this flawed experiment.

It's clear that without double-blind testing these "quality tests" are just so much mental masturbation.

I subscribed to Absolute Sound in 2011, I wanted to check it out. I certainly wouldn't renew my subscription. If you examine the whole "subjectivist" argument in the light of consistent logic, you conclude that these golden-ear reviewers believe that their OPINION is somehow better or "more true" than actual objective reality. This notion is absurd on the face of it, and for anyone who is capable of rational thought the subjectivist approach can be clearly seen to be either the result of genuine delusion or cynical dishonesty.

Go to the top of the page
+Quote Post
bug80
post Jan 19 2012, 10:38
Post #3





Group: Members
Posts: 403
Joined: 23-January 05
From: The Netherlands
Member No.: 19254



QUOTE (milosz @ Jan 18 2012, 20:47) *
What I would like to see is a bit comparison in a pair of buffers to see if two bit-identical files present exactly the same bitstreams to the DAC.

Question: are the contents of a Word-file different every time you open it on your computer? Do the words shuffle if you compress it using ZIP? If not, why would it be different for audio?
Go to the top of the page
+Quote Post
DonP
post Jan 19 2012, 15:00
Post #4





Group: Members (Donating)
Posts: 1473
Joined: 11-February 03
From: Vermont
Member No.: 4955



QUOTE (bug80 @ Jan 19 2012, 05:38) *
QUOTE (milosz @ Jan 18 2012, 20:47) *
What I would like to see is a bit comparison in a pair of buffers to see if two bit-identical files present exactly the same bitstreams to the DAC.

Question: are the contents of a Word-file different every time you open it on your computer? Do the words shuffle if you compress it using ZIP? If not, why would it be different for audio?


Bad example... the same as long as it's the same computer and same version of Word. Change those and things are often not quite the same. Things I've seen changed are the appearance of the dots on a bullet list, and how variable width fonts render.

Go to the top of the page
+Quote Post

Posts in this topic
- SoAnIs   The Absolute Sound FLAC article   Jan 7 2012, 04:19
- - testyou   FLAC is Lossless. What are you asking about?   Jan 7 2012, 04:41
- - A_Man_Eating_Duck   I struggle to read what you have posted any chance...   Jan 7 2012, 05:03
- - SoAnIs   I'm not asking anything. I'm describing th...   Jan 7 2012, 06:22
|- - Charlie Freak   I couldn't find that article on their website,...   Jan 7 2012, 07:12
|- - mjb2006   QUOTE (Charlie Freak @ Jan 6 2012, 23:12)...   Jan 7 2012, 07:31
- - MichaelW   The simple answer is that your father should no lo...   Jan 7 2012, 07:14
- - SoAnIs   Yep. The funniest bit is that none of the audiophi...   Jan 7 2012, 07:31
|- - zappalberto   QUOTE (SoAnIs @ Jan 7 2012, 07:31) Yep. T...   Nov 6 2013, 17:39
- - db1989   I’d compare this to shooting fish in a barrel—but ...   Jan 7 2012, 13:12
|- - Nessuno   QUOTE (db1989 @ Jan 7 2012, 13:12) I coul...   Jan 7 2012, 13:47
- - Speedskater   This is the third article in a four part series. ...   Jan 7 2012, 15:43
- - Apesbrain   "0 = Kazoo." Bummer. I play the kazoo ...   Jan 7 2012, 16:02
|- - SoAnIs   QUOTE (Apesbrain @ Jan 7 2012, 07:02) ...   Jan 9 2012, 21:40
- - ramicio   Is this the same article which claims that the spe...   Jan 9 2012, 21:58
|- - Porcus   QUOTE (ramicio @ Jan 9 2012, 21:58) Is th...   Jan 9 2012, 22:27
|- - ramicio   QUOTE (Porcus @ Jan 9 2012, 17:27) Of cou...   Jan 11 2012, 14:20
- - db1989   Yeah, from the same series. Sorry, I meant epic sa...   Jan 9 2012, 22:01
- - oldguy55   The article happens to be part 2 of a 4-part serie...   Jan 11 2012, 03:59
|- - krabapple   QUOTE (oldguy55 @ Jan 10 2012, 21:59) The...   Jan 11 2012, 06:03
|- - krabapple   QUOTE (krabapple @ Jan 11 2012, 00:03) An...   Jan 13 2012, 00:32
- - greynol   That reminds me about the time someone suggested t...   Jan 11 2012, 05:02
- - krabapple   QUOTE (SoAnIs @ Jan 6 2012, 22:19) 0 = Ka...   Jan 11 2012, 06:11
- - 2Bdecided   apparently part 3 is even better... QUOTE The art...   Jan 11 2012, 11:24
- - polemon   The fact that FLAC is lossless, and should always ...   Jan 11 2012, 11:28
|- - krabapple   QUOTE (polemon @ Jan 11 2012, 05:28) The ...   Jan 13 2012, 00:06
|- - polemon   QUOTE (krabapple @ Jan 13 2012, 00:06) um...   Jan 13 2012, 16:03
- - Satellite_6   Very amusing. Doesn't it seem like more than...   Jan 13 2012, 02:13
- - Wombat   I wonder if this things that TAS testers hear is t...   Jan 13 2012, 19:32
|- - PeterJvM   QUOTE (Wombat @ Jan 13 2012, 20:32) I won...   Jan 15 2012, 12:22
- - ron spencer   What is the motivation of this article? I must ad...   Jan 13 2012, 21:14
|- - slks   QUOTE (ron spencer @ Jan 13 2012, 14:14) ...   Jan 15 2012, 11:04
- - Squeller    If, with good gear and ears people could hear dif...   Jan 14 2012, 15:44
- - milosz   I agree that two bit-identical files should sound ...   Jan 18 2012, 20:47
|- - probedb   QUOTE (milosz @ Jan 18 2012, 19:47) What ...   Jan 19 2012, 10:24
|- - bug80   QUOTE (milosz @ Jan 18 2012, 20:47) What ...   Jan 19 2012, 10:38
||- - DonP   QUOTE (bug80 @ Jan 19 2012, 05:38) QUOTE ...   Jan 19 2012, 15:00
||- - bug80   QUOTE (DonP @ Jan 19 2012, 15:00) QUOTE (...   Jan 19 2012, 15:11
|- - Brother John   QUOTE (milosz @ Jan 18 2012, 20:47) What ...   Jan 20 2012, 12:02
- - Wombat   QUOTE (milosz @ Jan 18 2012, 20:47) I agr...   Jan 18 2012, 21:14


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st November 2014 - 03:46