Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Different Checksums (Read 12983 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Different Checksums

When I rip the same song from from different CD releases (i.e. the original release versus a rerelease) or even different pressings of the same CD release (i.e. the first pressing versus the second pressing) the checksums are different (I am using cdparanoia to rip CDs and sha1sum to generate checksums). What exactly is different between these digital files? Is some type of diagnostic information about the release and/or pressing embedded within each digital file? Would songs ripped from different CD releases and/or pressings sound exactly the same, or would they sound different?

On a similar note, back when Columbia House and BMG Music Service sold CDs in bulk for dirt cheap prices, would a song ripped from one of those CDs sound exactly the same as from the store-bought version, or would they sound different?

Thanks in advance! 

Different Checksums

Reply #1 –
A checksum is sort of a digital fingerprint of a file or string of data, even the slightest change will generate a completely different fingerprint.

What usually happens with compilation albums is that all the tracks are normalized to play back at the same volume. This alters the digital signature of the file completely, even though the music itself is untouched except for a slight change in volume.

On re-releases, there is often some remastering done, unfortunately it was very popular for a time to compress and amplify the hell out of everything in the name of "digital remastering". Luckily, there are also good remasters out there, but no matter if it's a good or bad remaster, it still changes the signal, and thus the checksum.

You can generally only expect to get the exact same checksum between two copies of an album, if both of those albums have the same label ID.

Different Checksums

Reply #2 –
Quote from: korndawg on
When I rip the same song from from different CD releases (i.e. the original release versus a rerelease) or even different pressings of the same CD release (i.e. the first pressing versus the second pressing) the checksums are different (I am using cdparanoia to rip CDs and sha1sum to generate checksums). What exactly is different between these digital files?


For different pressings, the difference is often merely left-shift or right-shift of the entire bitstream. That is why software like dBpoweramp and CUETools can test different offsets. You should definitely try a ripper with AccurateRip support - it will compare to what others have gotten. morituri, Rubyripper, (pird ?,) or Wine up a MS-Windows application (people have gotten CUETools to work under mono (using .wav only), but I do not know if the ripper works). I am not up-to-date on precisely which rippers can check across offsets though - http://wiki.hydrogenaud.io/index.php?title...n_of_CD_rippers may not be comprehensive and updated.

For different releases, it could very well be different masterings, which indeed are different.

Different Checksums

Reply #3 –
CUERipper works fine here (win32 wine prefix).

Different Checksums

Reply #4 –
KozmoNaut, thanks for your reply. I understand that compilation albums are often normalized and/or remastered, and I was asking specifically about checksum differences between rereleases and/or pressings of the same CD.

Porcus, when you say a simple left-shift or right-shift, you're talking about a few bits and/or samples that would not affect the sound in any way, correct? I previously calculated my CD drive's offset using Exact Audio Copy, but I am a Linux guy so I prefer to use cdparanoia with the "--sample-offset" parameter. Although cdparanoia does not support AccurateRip technology per se, it performs thorough jitter and error detection. Please let me know if I'm missing something.

Regarding Columbia House and BMG Music Service CDs, I found this dated yet extremely interesting article last night:

http://www.stereophile.com/features/55/index.html

Thanks in advance for any additional input! 

Different Checksums

Reply #5 –
Quote from: korndawg on
Regarding Columbia House and BMG Music Service CDs, I found this dated yet extremely interesting article last night:

http://www.stereophile.com/features/55/index.html


Please don't believe a single word of that article. The measurements he claims to do are sound enough, such as the cancellation test, but after that it completely veers into audiophile bullshit.

The measurements show no difference at all between the store-bought and record club editions, but he's so damned sure he hears a difference that he has to make up a reason. And it turns out to be the ever-nebulous jitter, currently the foil of any measurement-shunning audiophile. In a decade or two, it'll be something else that is blamed for any imagined differences in sound.

Different Checksums

Reply #6 –
Quote from: KozmoNaut on
Quote from: korndawg on
Regarding Columbia House and BMG Music Service CDs, I found this dated yet extremely interesting article last night:

http://www.stereophile.com/features/55/index.html


Please don't believe a single word of that article. The measurements he claims to do are sound enough, such as the cancellation test, but after that it completely veers into audiophile bullshit.

The measurements show no difference at all between the store-bought and record club editions, but he's so damned sure he hears a difference that he has to make up a reason. And it turns out to be the ever-nebulous jitter, currently the foil of any measurement-shunning audiophile. In a decade or two, it'll be something else that is blamed for any imagined differences in sound.


So in other words the Columbia House and BMG Music Service CDs contain exactly the same audio content as the store-bought versions? If hope that is the case because I have several Columbia House and BMG Music Service CDs from back in the day...

Different Checksums

Reply #7 –
Quote from: korndawg on
Porcus, when you say a simple left-shift or right-shift, you're talking about a few bits and/or samples that would not affect the sound in any way, correct?


Yes. 


Quote from: korndawg on
I previously calculated my CD drive's offset using Exact Audio Copy, but I am a Linux guy so I prefer to use cdparanoia with the "--sample-offset" parameter.


The reason why they could still differ said way, is that the masters are also written using different offsets. Per-drive offset correction in ripping only fixes offset differences between different drives used for reading.

The applications I mentioned run on penguinware. Myself I actually started using Windows to get dBpoweramp/fb2k ... of course you can use CUETools retroactively to verify with AccurateRip, provided that either you have cuesheets or all index marks are standard.

One more thing: there are many reasons to use .flac rather than .wav/.aiff, and one of them is the built-in checksum - which works on the (decoded) audio, so it is unaffected by tagging.

Different Checksums

Reply #8 –
Quote from: korndawg on
So in other words the Columbia House and BMG Music Service CDs contain exactly the same audio content as the store-bought versions?


Those are. And you can safely write off the jitterbug from Stereophile as nonsense (and even if it were true, it would not matter for the rips!).

But generally: because two CDs of the same title could be different masterings, so could a Columbia House and a retail version be - you do not know from the outset whether they used mastering X or mastering Y.

Different Checksums

Reply #9 –
But it is unlikely that Columbia House etc. would spend the resources on remastering or otherwise changing the sound, as that costs money, and the retail mastering job is perfectly adequate already. These clubs sell albums cheaper than retail, so there's absolutely no sense in them wasting money on unnecessary changes.

The whole "maybe they make deliberately more compressed versions for the 'lower class' stereos of their customer base" line of thought in the first paragraph is baseless audiophile elitism at its worst.

Different Checksums

Reply #10 –
Quote from: KozmoNaut on
But it is unlikely that Columbia House etc. would spend the resources on remastering or otherwise changing the sound, as that costs money, and the retail mastering job is perfectly adequate already.


Sure. But nothing says you are  less likely to get a different master from Columbia House, than in the record store. If customer asks "Why does [title] by [artist] from Columbia House sound different than the CD I already own?", then one should first make sure the customer knows enough to expect that there are different masterings around.

What I do not know anything about, is whether the CD clubs replace their masterings as the record company releases remastered editions. If not - or if "maybe not always" ...




Quote from: KozmoNaut on
The whole "maybe they make deliberately more compressed versions for the 'lower class' stereos of their customer base" line of thought in the first paragraph is baseless audiophile elitism at its worst.


Well ... I wouldn't call it "baseless" as it is essentially true, only if you point out the truly guilty part: lots of CD club releases sound inferior compared to the original, because the CD club releases are the very same masters as the ones the record companies crippled in the loudness war.

Different Checksums

Reply #11 –
Quote from: Porcus on
The reason why they could still differ said way, is that the masters are also written using different offsets. Per-drive offset correction in ripping only fixes offset differences between different drives used for reading.

I assume the "master" is the CD that is used as the source image for commercial pressing? Who generates the master? And they don't know how to correct for drive write offsets?

Quote from: Porcus on
One more thing: there are many reasons to use .flac rather than .wav/.aiff, and one of them is the built-in checksum - which works on the (decoded) audio, so it is unaffected by tagging.

I'm saving the ripped WAV files and then also compressing to FLAC as well. Is there a central location where these built-in FLAC checksums are stored?

Thanks so much for sharing your wisdom!

Different Checksums

Reply #12 –
Quote from: Porcus on
Well ... I wouldn't call it "baseless" as it is essentially true, only if you point out the truly guilty part: lots of CD club releases sound inferior compared to the original, because the CD club releases are the very same masters as the ones the record companies crippled in the loudness war.

As a neophyte audiophile I'm torn on remastering. Some remastered discs sound fantastic, while others are obviously brickwalled and compressed. Do you guys evaluate each remaster on a case-by-case basis, or do you always opt for the original recording? Are there any particular discs that stand out as the absolute worst remastering jobs? It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.

Thanks again for your help!

Different Checksums

Reply #13 –
Quote from: Porcus on
For different pressings, the difference is often merely left-shift or right-shift of the entire bitstream. That is why software like dBpoweramp and CUETools can test different offsets. You should definitely try a ripper with AccurateRip support - it will compare to what others have gotten. morituri, Rubyripper, (pird ?,) or Wine up a MS-Windows application (people have gotten CUETools to work under mono (using .wav only), but I do not know if the ripper works). I am not up-to-date on precisely which rippers can check across offsets though - http://wiki.hydrogenaud.io/index.php?title...n_of_CD_rippers may not be comprehensive and updated.

Would you recommend Exact Audio Copy? Exact Audio Copy supports drive offsets and AccurateRip, and supposedly runs flawlessly under Wine...

Different Checksums

Reply #14 –
Quote from: korndawg on
It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.

Such a database would probably be depressing to look at. It would just reflect the fact that even among "enthusiasts", anything remastered simply sells. Anything new sells, too; people think newer is better, and will say they like the newer sound because they can't imagine the record companies would ever let them buy something that sounds worse. People also generally prefer louder, brighter sounds...they'll rate highly any remastering that boosts the upper midrange, boosts the volume (even if it means killing the dynamic range), and widens the stereo image. Even people who seek out high-DR recordings may well say, in a blind and level-matched comparison, that they like the squashed remaster better because they like the updated EQ.

From personal experience, what sounded fine and bright when I was a teen sounds dull and muddy in my 40s. This is partly due to hearing loss, and partly due to the fact that I'm used to hearing modern music, which is mixed & mastered to be competitive EQ-wise, not just loudness-wise, and the trend has been to give certain kinds of sounds more "presence" and brightness than they would've had before the 1990s. Original masterings of pre-'90s recordings just sound like they're from another era, whereas newer remasters of that exact same material have a modern sound that fits better alongside contemporary music. Sometimes I like the updated sound of these remasters, sometimes I don't. I do consider the older masterings to be more authentic. On the other hand, what good is authenticity if I'm messing with the bass and treble anyway, listening on different gear, playing digital files instead of cassette tape, etc.? I'm still giving it an updated sound, to some degree.

Speaking of things from a bygone era, EAC isn't dead yet, but it's really falling off my radar. dBpoweramp and CUERipper are much simpler, slicker alternatives for secure ripping on Windows. However if they don't run well in emulators, you may be better off using EAC.

By the way, the CD drive "frame jitter" that cdparanoia copes with is not the same kind of sample-timing, wow & flutter-type of "jitter" that audiophiles normally talk about. What cdparanoia calls frame jitter is just inconsistent offset from read to read. Modern drives use Accurate Stream technology to minimize the chances of this happening, so the offset should always be the same on every read, and no detection and correction is necessary. However, I did have it happen to me once with a modern drive, so never say never.

Different Checksums

Reply #15 –
Quote from: korndawg on
Quote from: Porcus on
Well ... I wouldn't call it "baseless" as it is essentially true, only if you point out the truly guilty part: lots of CD club releases sound inferior compared to the original, because the CD club releases are the very same masters as the ones the record companies crippled in the loudness war.

As a neophyte audiophile I'm torn on remastering. Some remastered discs sound fantastic, while others are obviously brickwalled and compressed. Do you guys evaluate each remaster on a case-by-case basis, or do you always opt for the original recording? Are there any particular discs that stand out as the absolute worst remastering jobs? It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.


It really is a case-by-case thing, as it depends heavily on who's doing the remastering.

As a rule of thumb, I think that as long as the original release doesn't have any glaring faults, just go with that version if you can find it. A lot of modern remasters are overly compressed, and I've even seen a few driven into clipping for no reason at all.

But there are also some very good remasters out there, like the 25th anniversary edition of Machine Head, which is definitely my favorite. It's clean, well-balanced and makes everything slightly less muffled without destroying the original intent.

The best example I can find, which is kinda cheating, is "First Daze Here" by Pentagram. I've heard some samples of the original master tapes, and they are absolutely horrifically noisy. The remastering job done on them was an absolutely epic effort. Sure, the noise reduction is noticeable a lot of the time, but the snarl of the Rickenbacker bass and the bite of the snare drum is preserved so well that you cannot believe it came from those borderline-destroyed master tapes.

Done right, remastering can work wonders. Done wrong, it can destroy everything (I'm looking at you, Iggy Pop. The remaster of Raw Power was horrifyingly bad).

Different Checksums

Reply #16 –
Quote from: korndawg on
Is there a central location where these built-in FLAC checksums are stored?


In the file, you mean? It is part of the format, and e.g. vuplayer's audiotester.exe or foobar2000's file integrity verifier will decode, compute a checksum and compare to the one stored.

Quote from: korndawg on
It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.


http://dr.loudness-war.info/ offers some partial answers - not subjective "quality", but at least it gives some info on what is destroyed in the loudness war. Do not use it for vinyl, the metric is simply not suitable to compare a rip with vinyl noise to one without.
Regarding old masters, beware of pre-emphasis on a few titles.


Quote from: korndawg on
Would you recommend Exact Audio Copy? Exact Audio Copy supports drive offsets and AccurateRip, and supposedly runs flawlessly under Wine...


Has EAC implemented cross-pressing verification yet? If not, it is still a great piece of software, though I have to agree with mjb2006.  I settled with dBpoweramp seven years ago (and would have considered CUETools today).

Different Checksums

Reply #17 –
Quote from: Porcus on
Quote from: korndawg on
Would you recommend Exact Audio Copy? Exact Audio Copy supports drive offsets and AccurateRip, and supposedly runs flawlessly under Wine...

Has EAC implemented cross-pressing verification yet? If not, it is still a great piece of software, though I have to agree with mjb2006.  I settled with dBpoweramp seven years ago (and would have considered CUETools today).

Wine and Mono were misbehaving (shocking I know), so I took morituri for a test drive and it is FANTASTIC! It supports AccurateRip, and it automatically tags each file (previously I was manually tagging each file with Kid3). Is there any downside to morituri? I still have to carve out some time to fully RTFM, but I'm guessing that if the two fingerprints match (in this case they are both "9266803a"), then the rip successfully matches the AccurateRip database:

    Track  8: rip accurate    (confidence  75 of 200) [9266803a], DB [9266803a]

Also, I'm assuming that the more people verify the fingerprint, the higher the confidence rating. The only trouble I've encountered so far is that one disc was misidentified, specifically the Beatles 2009 Remastered Red Album Disc 1 was misidentified as the original 1993 version. During ripping three possible matches were identified:

    Matching releases:

    Artist  : The Beatles
    Title  : 1962–1966 (2009 remasters) (Disc 1 of 2)
    Duration: 00:31:03.772
    URL    : http://musicbrainz.org/release/1903f00b-51...77-ecf1835a34bb
    Release : 1903f00b-5131-48ee-9f77-ecf1835a34bb
    Type    : Compilation

    Artist  : The Beatles
    Title  : 1962–1966 (Disc 1 of 2)
    Duration: 00:31:00.239
    URL    : http://musicbrainz.org/release/301aa0fc-75...e2-74c3e9a7ebf7
    Release : 301aa0fc-75a7-3e55-82e2-74c3e9a7ebf7
    Type    : Compilation

    Artist  : The Beatles
    Title  : 1962–1966 (Disc 1 of 2)
    Duration: 00:31:01.973
    URL    : http://musicbrainz.org/release/d1ef3171-67...ce-f89c61bb2870
    Release : d1ef3171-678e-338d-a9ce-f89c61bb2870
    Type    : Compilation

However, unfortunately morituri picked the wrong disc:

    ARTIST:    The Beatles
    TITLE:      Love Me Do
    ALBUM:      1962–1966 (Disc 1 of 2)
    TRACKNUMBER:        1
    DATE:      1993-10-05
    MUSICBRAINZ_TRACKID:        1f518811-7cf9-4bdc-a656-0958e130f312
    MUSICBRAINZ_ARTISTID:      b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d
    MUSICBRAINZ_ALBUMID:        301aa0fc-75a7-3e55-82e2-74c3e9a7ebf7
    MUSICBRAINZ_ALBUMARTISTID:  b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d
    MUSICBRAINZ_DISCID: VTsA.WWKwyZD3AJSspUvKIRPKcE-

The fingerprints did match, so the rip was clearly successful, and I can just update the tags accordingly. But does this sort of disc misidentication thing happen often?

Thanks again for your help!

Different Checksums

Reply #18 –
Quote from: korndawg on
Also, I'm assuming that the more people verify the fingerprint, the higher the confidence rating.


As long as you have not submitted rips before, 2 or above will be practically equivalent in most cases. (If a certain CD has a confidence of 90 on all tracks but one, and 2 on that - well, it could be a manufactoring defect, but then there is most likely not "one correct way" to read it. If the odd track is the last one, then I guess the "lead out" should be the issue ... or whatever, I have forgotten the explanations I got, but it is not abnormal that the last track stands out with lower score.)



Quote from: korndawg on
The only trouble I've encountered so far is that one disc was misidentified


The metadata sources are not more reliable than what you see; freedb can be populated by anyone but corrected by no-one. In particular, if you disagree with whoever submitted the release and the year on whether the year should be of the original or of the reissue in question, you will have lots of retagging to do.

dBpoweramp can combine multiple metadata sources (and even uses Allmusic - which is prone to listing the year the release was first available in the US though...) AFAIK, EAC has started to get better sources as well, and CUETools is decent. Windows applications all of them.

Different Checksums

Reply #20 –
Quote from: Porcus on
If the odd track is the last one, then I guess the "lead out" should be the issue ... or whatever, I have forgotten the explanations I got, but it is not abnormal that the last track stands out with lower score.

It was due to a bug* in EAC which was fixed.  I think it was addressed before ARv2.  If so then there should only be discrepancies with the v1 results.

I tried to verify when Andre said that he fixed the issue, but couldn't.  I did find this:
http://www.hydrogenaud.io/forums/index.php...st&p=670259

(*) If you go back, you will see two separate issues.  The bug I'm mentioning here occurred when overreading was disabled (regardless of the capabilities of the drive).  This is not the same thing as when a drive incapable of overreading into the lead-out is configured as being able to overread, which is also problematic.

Different Checksums

Reply #21 –
Quote from: greynol on
The bug I'm mentioning here occurred when overreading was disabled (regardless of the capabilities of the drive).  This is not the same thing as when a drive incapable of overreading into the lead-out is configured as being able to overread, which is also problematic.


Ah! Not the same, but close enough to confuse an old memory. Thanks.

Different Checksums

Reply #22 –
Quote from: Porcus on
The metadata sources are not more reliable than what you see; freedb can be populated by anyone but corrected by no-one. In particular, if you disagree with whoever submitted the release and the year on whether the year should be of the original or of the reissue in question, you will have lots of retagging to do.

Just to clarify, the metadata sources are a completely different entity than the AccurateRip fingerprints, correct? In other words, as long as the AccurateRip fingerprints match with a confidence rating of at least 2, then the ripped audio files exactly match the CD? Correcting the occasional errant batch of metadata is no big deal, but I want to ensure the rips are accurate.

Thanks again for your help!

Different Checksums

Reply #23 –
Quote from: korndawg on
Just to clarify, the metadata sources are a completely different entity than the AccurateRip fingerprints, correct? In other words, as long as the AccurateRip fingerprints match with a confidence rating of at least 2, then the ripped audio files exactly match the CD?

Correct. And as pointed out by others: as long as you have not previously submitted the rip, it normally suffices to find a match to one other person's rip of the same title. (If the CD has manufactoring defects, the situation is somewhat different, but it is unknown whether anything can be done about those really.)

Notice that the term "fingerprint" is usually used for something different, namely for identifying music, not as a checksum for rips.


Different Checksums

Reply #24 –
Quote from: Porcus on
Correct. And as pointed out by others: as long as you have not previously submitted the rip, it normally suffices to find a match to one other person's rip of the same title. (If the CD has manufactoring defects, the situation is somewhat different, but it is unknown whether anything can be done about those really.)

If I verify AccurateRip checksums (by either ripping a CD or explicitly verifying checksums with morituri), does that automatically submit the rip? Or do users submit rips manually?

Also, is there any way to detect a manufacturing defect?

Thanks again!