IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
TAS at it again: bit-identical sources sound different. They just do.
krabapple
post Dec 31 2011, 21:13
Post #1





Group: Members
Posts: 2241
Joined: 18-December 03
Member No.: 10538



Seen today on the Hoffman forum:

"In the latest Absolute Sound magazine there is an article on computer audio which claims (get this):

2 bit identical files on a hard drive, played back with JPlay (which reads hard drive to memory, then plays from memory), sound different. One file was ripped at half the speed of the other, both with JRiver, both confirmed with AccurateRip"

I haven't read the TAS article, but I followed up online and found one of its authors responding to queries about it, on another forum. Here's what he says:

QUOTE
As one of the authors of this article, let me make a few comments. First we have taken some pains to describe our methods in detail in order to encourage readers to repeat our tests for themselves. Only then can they be in a position to dispute or confirm our findings. Opinions don't really count for much. Experimental results are more compelling.

In the quote cited by bramankp, we agree that one can make copies that are truly "identicle" but one cannot conclude that they therefore must sound the same. There are many factors that can change the sound between the information storage source and the DAC. For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different. Bits are bits all right, but time- and amplitude-based jitter mechanisms can account for many of the sonic differences we report.

To rossop: See Part 3 of our article in TAS #220 on FLAC. There are many sources and reasons why FLAC and WAV sound different, sometimes to a disturbingly large degree.


blink.gif

His other replies further down that thread are no more convincing,e.g

QUOTE
Lets make sure we are talking about the same experiment first. Using JRMC, under ripping options, there is a sub-menu for read speed: 1x, 2x, 4x, 8x, 16x (version 15 of the software). This is what we were testing. As an aside, it is an Interesting question to ask why the manufacturer includes this option in their software. Second, our hearing sensitivity and acuity was highly conditioned after listening to the same test tracks over and over again for 8 months. Third, I have over 30 years of professional industry listening training. Fourth, our auditioning system was optimized with respect to power cabling, digital cabling, AC grounding, vibration control, and speaker resolution. If your situation is not the same, then you have not replicated our experimental conditions exactly. I contend that if the differences are hearable on any system, they are not meaningless, but rather indicate a real effect regardless of whether they can be heard in all systems or just select systems. And the history of science and engineering includes many examples of disputed concepts being eventually proven correct. If you think the results in Part 2 are outrageously wrong, wait until you read what we have to say about FLAC files in Part 3.

On the other hand, there are criticisms that can be made of our procedures. For example, testing was usually conducted under single, not double blind conditions. Our results lack proper statistical rigor; generally speaking, we looked to replicate our results on two different systems and duplicate our tests at least once. It is also the reason why these results were submitted for publication to TAS and not the JAES. For the record, we did employ listening panels and numerous replications (at least 4) for what we felt were going to be the most controversial findings. In the case of the particular experiment we are speaking about, similar results were obtained some time ago with an independent computer and software and at an independent time. Had we tried to repeat everything we did in triplicate with at least 2 weeks intervals between listening tests, we probably would not have lived long enough to complete this study, or at least would no longer be married.




As we enter 2012, I want to say Thank you, HA, for being an antidote to audiophile diseases like TAS.

This post has been edited by krabapple: Dec 31 2011, 21:15
Go to the top of the page
+Quote Post
benski
post Dec 31 2011, 22:09
Post #2


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



Not that there's an ounce of sense in these people, but maybe someone should explain to them what disk cache is or a sound-card buffer? It sounds as if they believe that files are read off the hard drive "just in time" and go straight from the SATA controller to the DAC tongue.gif

This post has been edited by benski: Dec 31 2011, 22:09
Go to the top of the page
+Quote Post
Carledwards
post Dec 31 2011, 22:26
Post #3





Group: Validating
Posts: 30
Joined: 3-January 11
Member No.: 87003



It may be useful to remember that these magazines and writers, for the most part, have a vested interest in the sales of "audiophile" equipment and recordings. In light of that, their views and beliefs can be seen as self-serving rather than simply delusional. That's not to say that they're not delusional; just that many of them profit from the propagation of these "beliefs."
Go to the top of the page
+Quote Post
Speedskater
post Jan 1 2012, 00:39
Post #4





Group: Members
Posts: 139
Joined: 12-December 07
From: Cleveland, OH
Member No.: 49500



Another audio forum has a thread on the magazine article:

http://www.whatsbestforum.com/showthread.p...d-music-reprodu


--------------------
Kevin
Go to the top of the page
+Quote Post
Wombat
post Jan 1 2012, 02:39
Post #5





Group: Members
Posts: 1021
Joined: 7-October 01
Member No.: 235



They just are starting a new market indeed and breed a new generation of Audiophiles!
Now there isn´t new hardware and gear coming into their labs, it is Software!!
People that buy these things and believe these esotheric feelings will need such tests in the future for telling them how it sounds.
Everyone that sees evidence in these reports will always want to know how the new versions of his beloved software sounds in contrast to others.

Happy newe year btw. and a big Cheers to all of you. May the new year bring you health and clean ears smile.gif
Go to the top of the page
+Quote Post
Goratrix
post Jan 1 2012, 11:19
Post #6





Group: Members
Posts: 125
Joined: 5-August 08
Member No.: 56722



And another one of those threads over at the SH forum:

Where is the magic in a SHM disk?
Go to the top of the page
+Quote Post
db1989
post Jan 1 2012, 16:49
Post #7





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



Wow, this is absolutely terrible.

QUOTE
As one of the authors of this article, let me make a few comments. First we have taken some pains to describe our methods in detail in order to encourage readers to repeat our tests for themselves. Only then can they be in a position to dispute or confirm our findings. Opinions don't really count for much. Experimental results are more compelling.
As though they can appeal to the authority of science to make themselves sound trustworthy whilst flouting all of its proper procedures.

QUOTE
In the quote cited by bramankp, we agree that one can make copies that are truly "identicle" but one cannot conclude that they therefore must sound the same. There are many factors that can change the sound between the information storage source and the DAC. For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different. Bits are bits all right, but time- and amplitude-based jitter mechanisms can account for many of the sonic differences we report.
Notwithstanding that this assertion is mince, it is completely irrelevant. This study compared two files that were merely ripped at different speeds, then verified to be bit-identical before being played from the same disc through the same player. So, what possible relevance do any of these imagined phenomena have? Whence the difference? How can a difference emerge after bit-comparison, in response to a different ripping speed earlier in time? Do bits have memory like homoeopathic water? Just wow. How this writer thinks this is an adequate explanation is beyond me. (Then again, perhaps one can [pretend to?] believe anything when profit is at stake.)

QUOTE
Using JRMC, under ripping options, there is a sub-menu for read speed: 1x, 2x, 4x, 8x, 16x (version 15 of the software). This is what we were testing. As an aside, it is an Interesting question to ask why the manufacturer includes this option in their software.
Tinfoil hat nonsense. Something exists, so there must be a divine purpose for it, but one that is being suppressed by agenda-laden scientists! Nasty people!

QUOTE
Second, our hearing sensitivity and acuity was highly conditioned after listening to the same test tracks over and over again for 8 months. Third, I have over 30 years of professional industry listening training.
Appeal to authority, particularly distasteful in the second and more personal case, which just reads as Look at me! Please! and the obviously untrue implication that idiots and hacks cannot fashion comfortable and safe careers for themselves.

QUOTE
If your situation is not the same, then you have not replicated our experimental conditions exactly.
Malignant narcissism. We cannot be incorrect, naturally; so if you fail to agree with us, you are flawed. (Never mind that our experimental conditions are laughable.)

QUOTE
I contend that if the differences are hearable on any system, they are not meaningless, but rather indicate a real effect regardless of whether they can be heard in all systems or just select systems.
Whilst I doubt anything was heard, this is an astonishing abuse of reason.

QUOTE
And the history of science and engineering includes many examples of disputed concepts being eventually proven correct. If you think the results in Part 2 are outrageously wrong, wait until you read what we have to say about FLAC files in Part 3.
Turns out the Earth is actually flat after all!

QUOTE
On the other hand, there are criticisms that can be made of our procedures.
Where to begin?!

Again, just . . . Wow. I shall have to refrain from reading any more things like this (never mind the original article), for so many reasons.

This post has been edited by db1989: Jan 1 2012, 16:57
Go to the top of the page
+Quote Post
kraut
post Jan 1 2012, 18:58
Post #8





Group: Members
Posts: 227
Joined: 24-November 10
Member No.: 85965



QUOTE
for so many reasons.


One of them to keep your sanity. Nice take-down
QUOTE
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.
.

Where does he get the data from for such a contention - his behind maybe
Go to the top of the page
+Quote Post
krabapple
post Jan 1 2012, 20:05
Post #9





Group: Members
Posts: 2241
Joined: 18-December 03
Member No.: 10538



QUOTE (Speedskater @ Dec 31 2011, 18:39) *
Another audio forum has a thread on the magazine article:

http://www.whatsbestforum.com/showthread.p...d-music-reprodu



I see there that Clawson & Zeilig's article was rejected first by Stereophile for technical issues
Go to the top of the page
+Quote Post
JimH
post Jan 2 2012, 01:46
Post #10





Group: Members
Posts: 149
Joined: 14-July 02
From: Minneapolis
Member No.: 2588



While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

I don't agree with all that the authors said in the article, but I think they honestly tried to be objective (if you can apply that word to something as subjective as music). They spent about a year working on the article. I talked with Chuck Zielig 10 or 15 times and he was generally very precise and demanding about details.

I don't think that Stereophile rejected their aticle for technical reasons (or for any other reasons). I believe that the authors just got tired of waiting to hear from Stereophile and took their work to TAS.

I wish you a very Happy New Year!
Go to the top of the page
+Quote Post
kraut
post Jan 2 2012, 02:25
Post #11





Group: Members
Posts: 227
Joined: 24-November 10
Member No.: 85965



QUOTE (JimH @ Jan 1 2012, 17:46) *
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

That is exactly where the article seems to break down. The flawed methodology that actually ignores simple procedures that will ensure reproducible results that are worth further investigations, contentions that are just that without any evidence.
Opinions are dime a dozen and worthless without evidence.
I am of the opinion that stereophile is a worthless rag, and I have as evidence 12 years of collected bullshit in my attic. There is at least some evidence for my opinion.

QUOTE
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.

Without proper testing another bullshit statement that in this forum would not be permitted to be posted. Not even a cautious: might, no an outright will, as if they really had done any testing.
From what I understand - all they have done compare different sample rates, and nothing that gives them the right to the stated assertions.
Go to the top of the page
+Quote Post
db1989
post Jan 2 2012, 02:36
Post #12





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (JimH @ Jan 2 2012, 00:46) *
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

I don't agree with all that the authors said in the article, but I think they honestly tried to be objective (if you can apply that word to something as subjective as music).

Could you explain, generally or with particular reference to this article, “the importance of opinion, equipment, and hearing”? Which parts of it are worth agreeing with? I’m genuinely curious.

In objective terms—because your asserting the subjectivity of music as art is irrelevant to discussion of sound as a physical phenomenon—the test is fundamentally useless because it clearly was not performed blind.

However, its participants managed to go beyond the otiosity of sighted tests to the absurdity of concluding that bit-identical files, played in precisely the same way, could nonetheless sound different because of the speed at which they were ripped—as though this can somehow introduce a departure from their binary identity after this being verified, by what supposed means I cannot even imagine. The madness of this line of reasoning is profound.

At this point, I almost feel as though I must cut sighted tests some slack! Despite being devoid of objectivity, unreliable at best, and typically useless, they do not necessarily lead to this kind of nonsense: that requires a truly wild imagination, wholly unfettered by objectivity and reason.

Honestly, I’m not going out of my way to be disparaging, but I cannot conceive of any way to rehabilitate this ‘study’. It was based on unscientific methodology, and the biased results thus generated were presented as evidence of a totally ridiculous conjecture.

I really can’t believe that someone can hold ideas such as these, never mind sell them to consumers un(der)educated in the most basic means of evaluating claims presented to them. I’m genuinely saddened by it. (A shame, then, that this is one of the less harmful abuses of reason in our world.)
Go to the top of the page
+Quote Post
krabapple
post Jan 2 2012, 03:24
Post #13





Group: Members
Posts: 2241
Joined: 18-December 03
Member No.: 10538



QUOTE (JimH @ Jan 1 2012, 19:46) *
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

I don't agree with all that the authors said in the article, but I think they honestly tried to be objective (if you can apply that word to something as subjective as music). They spent about a year working on the article. I talked with Chuck Zielig 10 or 15 times and he was generally very precise and demanding about details.


But about statistics, not so much, apparently.

Excusing that by writing "this was aimed at Stereophile or TAS, not intended for a journal like JAES" , as the 'HiFi Doctor' Zeilig did, is pretty weak.

QUOTE
I don't think that Stereophile rejected their aticle for technical reasons (or for any other reasons).


Partly true -- it wasn't literally rejected . But it was offered to Stereophile, where it was not accepted outright, and that was due to 'technical issues' with the work....unless John Atkinson, Stereophile's editor-in-chief, was lying when he wrote this on WBF:

QUOTE
[The authors] offered this computer audio article to Stereophile but I felt there were some technical issues that needed to be addressed before the article could be published. Rather than address those issues, the authors withdrew it and offered it to TAS, where it appears to have been published as is.


This post has been edited by krabapple: Jan 2 2012, 03:56
Go to the top of the page
+Quote Post
Nessuno
post Jan 2 2012, 09:44
Post #14





Group: Members
Posts: 422
Joined: 16-December 10
From: Palermo
Member No.: 86562



QUOTE (kraut @ Jan 2 2012, 02:25) *
QUOTE
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.

Without proper testing another bullshit statement that in this forum would not be permitted to be posted.


Oh, let's go, this statement is the easiest to test and indeed something everyone experiences many times a day using computers.
It's just like when you open the same text document reading it from your old and evilishly fragmented hard disk or from a new and heavenly formatted usb memory stick: the very words and sentences you read are different, try and see it with your eyes! laugh.gif laugh.gif laugh.gif laugh.gif laugh.gif laugh.gif laugh.gif laugh.gif

Now the next audiophool quest will be: which is the best sounding filesysytem? And how much data journaling affects jitter? wink.gif


--------------------
... I live by long distance.
Go to the top of the page
+Quote Post
Destroid
post Jan 2 2012, 10:05
Post #15





Group: Members
Posts: 550
Joined: 4-June 02
Member No.: 2220



May I interrupt the logical arguments and ask if they bothered trying to burn it to vinyl first? Or better yet, burn it to water =D


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
smok3
post Jan 2 2012, 11:10
Post #16


A/V Moderator


Group: Moderator
Posts: 1728
Joined: 30-April 02
From: Slovenia
Member No.: 1922



QUOTE
Do bits have memory like homoeopathic water?

laugh.gif happy new year!

seriously: If bashing this kind of "tests" is the only thing that keeps hydrogenaudio still kicking, it is well worth it.
p.s. gotta find those golden cables now (left them on the roof freezing over night, so they will forget the music i was listening yesterday, hello!)

This post has been edited by smok3: Jan 2 2012, 11:21


--------------------
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung
Go to the top of the page
+Quote Post
john33
post Jan 2 2012, 13:45
Post #17


xcLame and OggDropXPd Developer


Group: Developer
Posts: 3760
Joined: 30-September 01
From: Bracknell, UK
Member No.: 111



It just beggars belief that anyone with even a basic understanding of digital audio could even consider asserting that bit-identical audio files, played on the same equipment could possibly sound different no matter how the audio was originally ripped. I can't even imagine the mind set that would permit you to consider that there was anything to test, nevermind persuade yourself, and others that there was any difference to be considered. That he even states that he has "over 30 years of professional industry listening training" only serves to indicate how flawed that training must have been.

Anyway, just my 2 cents. I've been trying to resist making any comment here but have finally given in! wink.gif

Happy New Year, everyone. smile.gif


--------------------
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/
Go to the top of the page
+Quote Post
Stereoeditor
post Jan 2 2012, 15:02
Post #18





Group: Members
Posts: 227
Joined: 24-April 09
Member No.: 69239



QUOTE (JimH @ Jan 1 2012, 20:46) *
I don't think that Stereophile rejected their aticle for technical reasons (or for any other reasons). I believe that the authors just got tired of waiting to hear from Stereophile and took their work to TAS.


While it it is true that it took me longer than the authors liked to go through their article - it was 16,000 words - I felt there were methodological problems and technical misstatements that would need to be addressed. I discussed these issues with Chuck Zeilig last May, but his reaction was to withdraw the offer to have the article published in Stereophile. It has subsequently been published in The Absolute Sound in its original form, as far as I can tell from the first two installments.

If you check with Chuck, he can confirm that the discussion with him, in which I let him know of some of my reservations, took place on Thursday May 26 and his withdrawal was on Friday July 1.

John Atkinson
Editor, Stereophile
Go to the top of the page
+Quote Post
JimH
post Jan 2 2012, 17:39
Post #19





Group: Members
Posts: 149
Joined: 14-July 02
From: Minneapolis
Member No.: 2588



Hi John,
I can understand why it took a long time to review the article. It is big. I also understand that it might have needed your editing attention. In the end, it did get published and it will probably continue to be scrutinized and criticized for some time to come.

I know that Stereophile will continue to give computer audio close attention and I look forward to reading more there.

For those who haven't read the article, it's not just about how some zeroes and ones are better than others. I think it will help convince mainstream audiophiles that computer audio has come of age. Of course, if you're a forum member here, you've known that for a while.

No matter whether you agree with what audiophiles believe, I hope you will find it in your hearts to give them credit for their passion.

Jim Hillegass
CEO, JRiver
Go to the top of the page
+Quote Post
db1989
post Jan 2 2012, 17:57
Post #20





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (JimH @ Jan 2 2012, 16:39) *
In the end, it did get published and it will probably continue to be scrutinized and criticized for some time to come.
But probably not to the degree that it deserves, especially not in its original location of TAS. If anything, it may just help to further entrench established (and baseless) prejudices against particular aspects of digital audio, and to add to the ranks of the misled (and the coffers of the snake-oilers).

QUOTE
For those who haven't read the article, it's not just about how some zeroes and ones are better than others. I think it will help convince mainstream audiophiles that computer audio has come of age. Of course, if you're a forum member here, you've known that for a while.

No matter whether you agree with what audiophiles believe, I hope you will find it in your hearts to give them credit for their passion.
Computer audio has come of age because nonsense like the above (which is merely one facet of the article, by the way) is able to be published and believed? Audiophiles deserve credit because they are able to say things like this and find an audience? Forgive me if I cannot make the logical connection in either of these examples.

Also, as per my above post #12, which you seem to have overlooked, I am still curious about how you would defend this article (beyond what seems to be your idea that any talk about audio is good talk about audio).
Go to the top of the page
+Quote Post
Deep_Elem
post Jan 2 2012, 18:47
Post #21





Group: Members
Posts: 120
Joined: 14-April 04
Member No.: 13473



So when I copy audio files from one part of my system to another, e.g. a different hard drive, should I copy them at half speed or full speed in order to get the best sound?

Yes I'm kidding.
Go to the top of the page
+Quote Post
Juha
post Jan 2 2012, 19:03
Post #22





Group: Members
Posts: 453
Joined: 14-February 07
From: EU-FIN
Member No.: 40610



QUOTE (Deep_Elem @ Jan 2 2012, 21:47) *
So when I copy audio files from one part of my system to another, e.g. a different hard drive, should I copy them at half speed or full speed in order to get the best sound?

Yes I'm kidding.



Loopback cable (DA-AD conversion) is worth to try (if those are CD rips). laugh.gif


Juha
Go to the top of the page
+Quote Post
Speedskater
post Jan 2 2012, 19:10
Post #23





Group: Members
Posts: 139
Joined: 12-December 07
From: Cleveland, OH
Member No.: 49500



The same two authors did publish a 7 page article in the June 1986 issue of Stereophile. The Stereophile editors did have a dis-claimer about the authors conclusions. The article's premise was using commercial recording of the same music in LP record, CD and cassette tape, determine which play-back system sounded best.


--------------------
Kevin
Go to the top of the page
+Quote Post
Wombat
post Jan 2 2012, 19:43
Post #24





Group: Members
Posts: 1021
Joined: 7-October 01
Member No.: 235



Me doesn´t know much but there are many that know nothing and use excactly such publications to argue.
I just was in contact with some audiophiles at diyaudio.com that use terms and examples of such reviewers and marketing to argue in the forum all the time. They just repeat what they read without understanding anything of it, leave alone being able to conclude anything themself.
These individuals must be leaded and most likely highly weclome such pamphlets as another prove of their "knowledge"
I learned the last days at diyaudio that 1-bit DSD sounds better because it creates REAL samples of music and linear filters are the most destructive thing that can be done to audio. I was prompted to practice, learn and finaly to use my ears!
Go to the top of the page
+Quote Post
JimH
post Jan 2 2012, 20:15
Post #25





Group: Members
Posts: 149
Joined: 14-July 02
From: Minneapolis
Member No.: 2588



QUOTE (db1989 @ Jan 2 2012, 10:57) *
Audiophiles deserve credit because they are able to say things like this and find an audience? Forgive me if I cannot make the logical connection in either of these examples.

They're saying what they believe to be true. It's up to you to accept or reject it. As I said, I don't believe it all.

QUOTE
Also, as per my above post #12, which you seem to have overlooked, I am still curious about how you would defend this article

I'm not defending the entirety of the article. I believe they have a right to a fair and objective analysis of their work. I think some of the people who are the most negative about it may not have read it.

QUOTE
(beyond what seems to be your idea that any talk about audio is good talk about audio).

I don't think I said that. As I've said, I'm a scientist and an engineer. I like proof.

On our forum, we occasionally encounter some questionable opinions about audio. Here's an example: "... ape sounds better than flac ..."

On some forums, this might be out of bounds, but we let this kind of discussion continue for a while. It generally blows itself out.

This post has been edited by JimH: Jan 2 2012, 20:16
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st September 2014 - 20:10