Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Drive offset and its affect on last sector during CD rip (Read 7843 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Drive offset and its affect on last sector during CD rip

Hello. As per the title I have a few questions regarding drive offset and its affect with ‘last sector’ handling during CD ripping.

Some years back I ripped my CD collection using EAC to single image (album) FLAC with cuesheet. Not sure why but a week or so back I decide to have a look at CueRipper and grabbed a random CD (Pink Floyd – The Final Cut). I actually hit an issue with CueRipper with it raising an exception during gap detection (as per another thread) but that’s not the reason for the post.

What I discovered was that the MD5 within the FLAC was different to my earlier ripped master. My initial reaction was along the lines of what the...

As it happens I have 2 notebooks and 2 PC. The PC that was used for the ripping was retired but I managed to resurrect it and repeat the rip and it got the same MD5 as what was in the original (so the same system got the same MD5).

I then tried all others systems and found a relationship between the offset. Somewhat by chance, two systems have an identical offset of +102 and two have an identical offset of +667. Each system with the same offset results in the same MD5. In all cases, AccurateRip results (both AR1 and AR2) following ripping are fine.

Next step was to compare WAV files so I decoded each of the FLAC and compared within EAC. Both files are bit identical right up to what I would call last sector with the variation occurring in last 0.013’ of the WAV file (so last 1/75).

Upon reflection I expect this is probably quite normal and I note that AccurateRip does not use last 5 sectors (2940 samples) for its calculation. I presume this is intentional due to drive variation and the like.

I guess the question is should I be concerned about this or is this a case of OCD getting the better? Is there any ripping means that overcomes the variation of offset and drive capability that will result in bit identical WAV? Keep in mind I’m asking about the very last samples; I expect everything up to then to be bit identical which is the purpose of the offset value used with ripping.

It does seem to me that consistency will only be achieved where the drive has an absolute offset of less than or equal to 588. Any offset larger than that will result in a variation subject to EAC settings, offset value and drive capability when ripping the last samples.

Just out of curiosity, the FLAC MD5 for the album as above comes out as per below. I’d be interested in the value that others have for comparison as it will confirm my thinking to the cause and maybe as to which is the ‘better’ image copy. Thoughts?

Offset +102 FLAC MD5 9f9bbafb2937a6466c6ca54ab472b598
Offset +667 FLAC MD5 1111c22a05506dc693b57dc83cca727e

Drive offset and its affect on last sector during CD rip

Reply #1
I think this has to do with overreading, since both drives have a positive offset correction value they both need to overread into the lead-out in order to completely extract the audio data of the last track. Note that apparently some drives incorrectly report their overread capability, and that a drive's ability to overread is independent of offset size.

My advice is to not worry about this at all, but I suppose that if you wanted to find out which is the "better" rip you could try ripping that CD in a drive with negative read offset correction which would be able to completely extract the last track, and then compare it with your previous 2 rips to see which (if any) of those is the right one.

FWIW, my drive has a +48 offset, and in some CDs I get bit-identical results across EAC, CUERipper amd dBpoweramp, whereas in others I encounter the same problem as you, the last track differs in a variable amount of samples between the EAC and CUERipper/dBpa (these 2 are always bit-identical) rips.


Drive offset and its affect on last sector during CD rip

Reply #2
kudabird, your MD5 hashs tell us nothing, if you don't also tell us the exact FLAC settings you used. Only then will others be able to compare MD5 with yours.
lame -V 0

Drive offset and its affect on last sector during CD rip

Reply #3
kudabird, your MD5 hashs tell us nothing, if you don't also tell us the exact FLAC settings you used. Only then will others be able to compare MD5 with yours.


The MD5 I'm referring to is within the metadata visible from metaflac. Happy to be corrected but I understand this is a calcualation based on the input file and is setting independant. Assuming the same CD pressing, anyone with a drive offset less than or equal to 588 should have a common MD5... in theory.

Drive offset and its affect on last sector during CD rip

Reply #4
Is there any ripping means that overcomes the variation of offset and drive capability that will result in bit identical WAV?

Yes, it is referred to as the ability to overread into the lead-out.

It does seem to me that consistency will only be achieved where the drive has an absolute offset of less than or equal to 588. Any offset larger than that will result in a variation subject to EAC settings, offset value and drive capability when ripping the last samples.

I don't see why this would seem to you since it is a false notion.  That you assume there is some theoretical basis for it is a bit perplexing.

Drive offset and its affect on last sector during CD rip

Reply #5
Is there any ripping means that overcomes the variation of offset and drive capability that will result in bit identical WAV?

Yes, it is referred to as the ability to overread into the lead-out.

It does seem to me that consistency will only be achieved where the drive has an absolute offset of less than or equal to 588. Any offset larger than that will result in a variation subject to EAC settings, offset value and drive capability when ripping the last samples.

I don't see why this would seem to you since it is a false notion.  That you assume there is some theoretical basis for it is a bit perplexing.


A sector size is 2352 bytes or 588 samples. If a drive does not have to extend into the lead-out, in other words the required samples can be read within the last sector then yes, I would expect a level of consistency with the resultant WAV. Beyond that you're further exposed to settings, drive capability and the like which will be numerous.

And I have tested by reading into the lead-out but then you start getting AR failure for want of a better description with the last track. Presumably no one (or at least very few) read into the lead-out during ripping or drives/ripping tools are not setup/capable which has ultimately influenced the AR database confidence values.

I'm just trying to ensure I get (as best one can) accurate extraction with AR pass. If I can also get confirmation from a fellow member via check of the MD5 within the metadata for the CD title as above all the better. I do however think I've research enough now know that my system with the drive offset of +102 are likely the more 'accurate'; that is, all tracks pass AR, correct length, least amount of padding/silence added etc.

Drive offset and its affect on last sector during CD rip

Reply #6
A sector size is 2352 bytes or 588 samples. If a drive does not have to extend into the lead-out, in other words the required samples can be read within the last sector then yes, I would expect a level of consistency with the resultant WAV.

Again, your notion is completely baseless and without merit.  The resolution of offsets are in samples not "sectors".  Drives with different offsets see "sectors" differently; IOW, the data within each "sector" is also shifted.

Both your drives require overreading from the lead-out if you wish to grab every last sample in the range defined by the EAC reference.  Your other option is to null the last 667 samples from all the rips (an additional 565 samples of data will be lost from the ends of the last tracks ripped with the drive with the +102 read offset correction) in order for the data to be the same regardless of the drives.

Beyond that you're further exposed to settings, drive capability and the like which will be numerous.

I'm very well acquainted with the details, thanks.

And I have tested by reading into the lead-out but then you start getting AR failure for want of a better description with the last track.

Only because EAC doen't do a very good job of preserving the samples the drive can extract when it is configured to overread when the drive really can't.

Presumably no one (or at least very few) read into the lead-out during ripping or drives/ripping tools are not setup/capable which has ultimately influenced the AR database confidence values.

Has ultimately influenced? Can you provide some data illustrating this?  I'd be very interested in seeing it.

I realize that I'm giving you a hard time, but that's what I typically do when I read posts where someone pretends to know far more about what is being discussed than he really knows, et cetera and the like.

PS: The only DAE programs that can submit to the AR database are EAC and dBpoweramp.

Drive offset and its affect on last sector during CD rip

Reply #7
It with some trepidation that I post again but what I’m attempting to understand is causes for the variation in ripping result. I believe it is correct to say that where the offset of the drive is less than or equal to 588 then reading of lead-out is irrelevant; the drive should be perfectly capable of reading the last 2352 bytes or 588 samples.

When I enable reading into the lead-out (EAC), the ending portion of the WAV file is different again. Whether the drive actually went into the lead-out or failed and returned garbage or silence is academic; the content of the WAV ending is different. In my case, it is different enough that the last track did not validate as accurate with AR. So in my case, setting for reading into the lead-out is not a solution if you then can’t use the results with AR.

It is on that basis that I made the comment about AR database accuracy. Reading back perhaps I wasn’t clear. What I meant was that if everyone suddenly started reading into the lead-out or made other settings changes that affected the final portion of the WAV, there would never be a consistent AR value (for last track) to compare with. This I expect is the very reason AR does not use last 5 sectors for it AR calculation.

Your reply suggests that EAC is buggy or at least poorly handles reading of the ending of the last track's sector and lead-out. If EAC is so bad with last track I’m surprised it’s not a more widely discussed topic. How does one know they have the most accurate extraction possible if no one has taken the time to question and/or compare the WAV CRC or FLAC generated MD5, as AR alone does not vouch for the full WAV? In your experience, is dbPowerAmp any better with its extraction? I see it has a trial option so no harm to take a look and compare it’s behavior across the systems. I’ve also got some tools tucked away that will RAW read the CD including TOC so should be able to check if the drive(s) can read the lead-out and compare the extracted sectors to the LBA within the TOC.

Finally, let me put a different slant on it; what if those last samples contained music? What if an error occurred reading those samples but it was outside of the range that AR uses for it confidence value? This is why when I rip I compare CRC and/or MD5. Of course, a consistent checksum only confirms I read the CD accurately twice or more; it doesn’t compare against anyone else. I could still have bad and/or different samples not tested by AR. I had never given this thought until a different system gave a different MD5 and yet still passed AR.

Anyway, I think I’ve probably laboured enough; time to test some alternatives. Thanks for the reply as it has helped.

Drive offset and its affect on last sector during CD rip

Reply #8
While I can lead you to water...

I believe it is correct to say that where the offset of the drive is less than or equal to 588 then reading of lead-out is irrelevant; the drive should be perfectly capable of reading the last 2352 bytes or 588 samples.

You're still very much incorrect whether you choose to believe it or not.  Drives with different offsets see the frames differently whether they be the first, last or something in between as the samples in each and every one of them are different.  As soon as you configure a drive with any positive read offset correction (1, 587, 588, 589 or any other value) and tell EAC that it can overread, you are instructing the drive to read a frame or frames in the area it deems to be the lead-out.  This is the last time I will attempt to explain it.  If either I fail to make sense or you choose to remain ignorant, so be it.

What I meant was that if everyone suddenly started reading into the lead-out or made other settings changes that affected the final portion of the WAV, there would never be a consistent AR value (for last track) to compare with. This I expect is the very reason AR does not use last 5 sectors for it AR calculation.

In the case where data for any particular disc ID already exists, any unique value submitted to AR is withheld from the public record until they are validated by a submission from a different user ID.  FWIW, it actually is possible for AR to contain submissions from people ripping the same pressing with the same offset correction using a drive that is incorrectly configured to overread.  I simply wanted you either to demonstrate that you understood how it could occur or provide evidence indicating that it had occurred.

Your reply suggests that EAC is buggy or at least poorly handles reading of the ending of the last track's sector and lead-out. If EAC is so bad with last track I'm surprised it's not a more widely discussed topic. How does one know they have the most accurate extraction possible if no one has taken the time to question and/or compare the WAV CRC or FLAC generated MD5, as AR alone does not vouch for the full WAV? In your experience, is dbPowerAmp any better with its extraction? I see it has a trial option so no harm to take a look and compare it's behavior across the systems. I've also got some tools tucked away that will RAW read the CD including TOC so should be able to check if the drive(s) can read the lead-out and compare the extracted sectors to the LBA within the TOC.

I'd wager to guess that most people simply configure their drives correctly and move on and figure all bets are off if they don't.  FWIW, I'm fairly certain the Red Book specification does not call for the use of logical block addressing, nor does it use the term sectors, for that matter.  If it had, I'm just as certain that we wouldn't be talking about offsets and overreading.  Regardless, have fun with your experiments.

Finally, let me put a different slant on it; what if those last samples contained music? What if an error occurred reading those samples but it was outside of the range that AR uses for it confidence value?

If the only differences fall outside the data covered by AR then the AR hashes will be the same.  This was put into place so that AR could handle drives with different offsets that could not overread.  I believe spoon didn't realize that an incorrectly configured overreading option could complicate matters, at least this is the impression I got when we discussed it a few years back.

This is why when I rip I compare CRC and/or MD5. Of course, a consistent checksum only confirms I read the CD accurately twice or more

Replace "accurately" with "consistently"; they are not necessarily synonymous.

Drive offset and its affect on last sector during CD rip

Reply #9
Hello. I actually started a reply but deleted my draft for reasons that are somewhat obvious, then though why not. As is often the case, finding the right keywords for search is critical and I have now found an answer to my question. It’s from the dbPA forum some 4 years ago and answered by Spoon as it happens.

http://forum.dbpoweramp.com/showthread.php...ull=1#post76372

I actually had a genuine question; a desire to find out why the FLAC checksum with my rip was different across systems. Is that not a worthy goal? To do that I did have to test. Given this is HA which expects testing and evidence and not anecdote I find your comment ‘"Regardless, have fun with your experiments"’ to be quite out of order.

As you pointed out, it does seem that EAC resulting WAV file can and does differ depending on the drive offset and drive capability. I can replicate its behavior including to the point of AR failure even though the rip is valid. That might be common knowledge for you or you may well have be speculating about EAC but it is news to me and I expect others as well.

Regarding your comment "‘I'd wager to guess that most people simply configure their drives correctly and move on"’. Are you sure? I say the contrary is equally likely. Given the EAC behavior, that it has a large install base and that large offset drives are somewhat commonplace, the fact that it does not appear to be a discussed topic tells me people are unaware and are not testing their rips. How would they know if they have their configuration setup correctly including for AR submission?

Finally your comment "‘I'm fairly certain the Red Book specification does not call for the use of logical block addressing, nor does it use the term sectors, for that matter"’. Agreed, but we are not discussing playback. The topic is DAE and the terms sector and LBA are commonplace in that context and you would know that.

I actually think I had a legitimate question and I tried to present it as accurately as I could; I’ll let other's judge what I had posted. I participate in forum here in Australia and I must say it’s unusual to see a question so sidetracked for semantics so early on. Is that what I can expect from HA in the future? In part I registered as I saw a recent post regarding command line manipulation of FLAC metadata and as it happens I have some VBScript that I wrote for that purpose, as well as transcoding with retention of metadata. They can be used as-is but easy enough for the average Joe to customize though now I’'m not sure I can be bothered.

Drive offset and its affect on last sector during CD rip

Reply #10
Actually I would have to expand on what I said 4 years ago, if a drive cannot overread and the ripper fills those missing samples with zeros (and they are zeros on the disc) then you would get a consistent CRC, but not for the case where there are samples right to the end (typically it could be background silence, or a badly mastered disc where the audio cuts out short).

The issue here are rippers which try to read TOC end - X, where X happens to be number that the ripper is using (ie track len / reading LBA block size), the failure to read will not only be a failure of the last sector, but the whole lot, potentially 30 sectors which is then silenced by the ripper. dBpoweramp will on a failed read of last sectors, scale back the request so it falls back within the LBA range and zero the missing part.

I would have to actually say, for most people testing such is difficult, because you would have to have a CD which is audio to the end, or make one. Also different drives do different things, I am sure I have seen a drive which was happy to pretend to overread, as long as there were silence beyond the end, but as soon as there is real audio it failed...

Drive offset and its affect on last sector during CD rip

Reply #11
I find your comment ‘"Regardless, have fun with your experiments"’ to be quite out of order.

I was being quite sincere.  I did not in any way want to discourage you from experimenting in order to gain further understanding as to what is going on.

From my point of view, the reason why your flac md5sums of the raw audio data were different depending on the drive is quite trivial; the data is different and the obvious explanation is that one drive can read farther to the end of EAC's calibrated range than the other and the samples in that area were non-null.  This is hardly new territory (random link).  While it may not have been covered much recently, it has been discussed with a fair amount of regularity for over a decade now.  You're right in that you need to use the right search terms and those that are hyphenated "lead-out" might not show up easily, but I made sure to use them in my responses to you in hopes that you would conduct the research on your own.  FWIW, of the over 8000 posts I have logged here, a sizable chunk is dedicated to DAE in general and ticky-tack things like overreading and AR in particular.

or you may well have be speculating about EAC but it is news to me and I expect others as well.

It is usually very obvious when I speculate; I either use qualifiers or I will come right out and say it.

How would they know if they have their configuration setup correctly including for AR submission?

People around here either know the settings or follow our wiki or other online guides.  EAC has an internal test for overreading and those who don't bother typically leave it disabled (which is the default, BTW).  For those who dare to change it, there is no problem until a disc is read that has non-null samples in the area deemed by the drive as the lead-out, resulting in an error.  I don't think it unreasonable to assume the next course of action is to investigate the problem online where a half-way thoughtful search will give plenty of good responses telling the user to disable the overreading setting, with a large portion of the results coming from this forum or from digital-inn.de, the official EAC support forum.  Those interested in investigating further are generally most successful when they run tools to spot differences (EAC and foobar2000 come to mind) and/or load tracks into a wave editor.

The topic is DAE and the terms sector and LBA are commonplace in that context and you would know that.

Sector, yes; LBA? No; not at all.

Using the term LBA with CDDA basically raises a red flag indicating a fundamental misunderstanding of CDDA.  I suspect it is this very misunderstanding that is the reason why you're having difficulty.

Through experimentation you will hopefully discover this, since you appear unwilling to listen to what I have to say.  FWIW, I strongly urge you to use a wave editor as one of your primary tools!

I participate in forum here in Australia and I must say it’s unusual to see a question so sidetracked for semantics so early on.

I wouldn't be so quick to dismiss being told that LBA has no place in CDDA as semantics.

PS: Reading spoon's reply I guess we can add another term being misapplied to DAE.  They include, but are not limited to: jitter, sectors, AccurateRip "CRC" and now LBA.  Joy! 

Let this discussion serve as an example about how attempting to take a term like LBA literally can be confusing:
A sector size is 2352 bytes or 588 samples. If a drive does not have to extend into the lead-out, in other words the required samples can be read within the last sector then yes, I would expect a level of consistency with the resultant WAV.

Drive offset and its affect on last sector during CD rip

Reply #12
Thanks Spoon. And I used dBPA and compared to EAC regardless of setting I get a consistent rip with matching MD5 and checksum for AR. So chalk one up for dBPA I guess. To be clear, the very last sample are different betwen the systems with +102 and +667 offset but I understand the reason now. Cheers.

Drive offset and its affect on last sector during CD rip

Reply #13
I was all set to concede the use of LBA was misplaced and call 1-0 to you, and then I too saw the Spoon reply ;-)

Drive offset and its affect on last sector during CD rip

Reply #14
Again, there is no LBA in Red Book.  To use the term out of ignorance, laziness or some other excuse is unfortunate.  Addressing in Red Book is fundamentally different than Yellow(?) Book and other storage formats where LBA is actually used.  An audio CD player scans the subcode that rides along with the audio data in order to determine where it is and in which direction to seek.  The TOC simply provides track start times.

Regarding dBpa gracefully handling overreading, I dug this up for you:
http://www.hydrogenaudio.org/forums/index....showtopic=59643

I've uncovered quite a few topics on overreading looking for it. 

So can you find other people commonly considered authorities on the subject using LBA besides spoon?

Drive offset and its affect on last sector during CD rip

Reply #15
So can you find other people commonly considered authorities on the subject using LBA besides spoon?


I'm up for a challenge ;-) Why you yourself provided the definition of LBA http://www.hydrogenaudio.org/forums/index....st&p=529040. That didn't take long; can we call it even ;-)

I do think we're done here. I now know the WAV will be different where the offset is different to the extent I have and I've learnt about EAC and its behaviour with one particualr drive that I have. That's more than I knew earlier so thanks.


Drive offset and its affect on last sector during CD rip

Reply #16
LOL.  Well I do have egg on my face and humbly admit to being very wrong for using LBA in that post (and any others if they exist).

You do now understand that drives that cannot overread which have offset corrections of +6, +12 and +102 will give different results when ripping the last track of a disc that contains PCM data which is non-null up to the very end, correct?

BTW, thanks for considering me an authority on the subject.

Drive offset and its affect on last sector during CD rip

Reply #17
All good. And I've got to say I too had a bit of a chuckle. Not only do you use the term you provide the damn meaning. Gold :-)

Drive offset and its affect on last sector during CD rip

Reply #18
There is nothing wrong with using the Term LBA and CD ripping - CD ripping is addressed (through SCSI commands) using either MSF (minute seconds frames) or LBA (logical block address), EAC AFAIK uses MSF internally, dBpoweramp uses LBA addressing.

For reference to CDDA and LBA have a look at any of the SCSI Multimedia command documents (mmc-2, mmc-3, etc)

MSF was designed so the reading location of an audio CD can go straight to the display of a cheapo audio player, but any CD computer drive must be able to translate internally between LBA + MSF for all relevant SCSI commands (TOC read, read raw, etc)...and there are drives which even fail this simple task (a TOC read in MSF returns different times than TOC read in LBA, dBpoweramp reads both in the debug log, so we spot this time to time).

LBA is actually the better way of addressing, as the chance of bugs is greatly reduced, for example subtracting 135 frames from a LBA address of 12345 is a simple minus, where as subtracting 135 from a 20/3/45 address, is not to easy.

Drive offset and its affect on last sector during CD rip

Reply #19
Again(!) Red Book makes no mention of LBA.  CDs and "cheapo players" existed long before optical computer drives.  As anyone who has dabbled in DAE knows, the marrying of Red Book and the PC was not all that straight forward.  Applying the idea of LBA as if it was leads to problems as evidenced by the idea that a player should be able to retrieve an entire "sector" of audio from a disc even if part of that "sector" lies in an area the drive considers out of bounds.

So while you may interface with a drive at an LBA level, it doesn't reflect the reality of what is going on beyond that point.  Once we discuss offsets and overreading, guess what, we have gone beyond that point.

Drive offset and its affect on last sector during CD rip

Reply #20
Elaborating from Spoon to make it more clear for others, it seems to be a flaw in EAC's method of always reading (requesting) blocks of sectors upto 64kB per drive read command.  When the rip nears the end of a CD EAC's method of reading can overshoot (past) the last drive readable sector of the CD which will result in an error and then all prior sectors which were readable in that block is dropped by the drive or returned as zeroes.

Quote
Sector, yes; LBA? No; not at all.
LBA (Logical Block Addressing) is just a fancy term for sector number and you did say Sector is alright with you, also drives can only request for sectors (2352 bytes of main) as there is no drive command to request for single byte or sample.

Drive offset and its affect on last sector during CD rip

Reply #21
I thought this thread was about cd ripping...not audio only cd transports. Read book is only half of the equation, as the interface to the CD drive is the other half when it comes to cd ripping.

mmc-1 has been around for the last 16 to  17 years! and LBA has always been part of it. As for talk of convergence of computers and cd drives, there were specific vendor audio ripping modes, 17 years ago I was working with them, they were not pretty, they should be seen for what they were, non standard vendor hacks, mmc was the first standard. Talking about these hacks as though they have relevance today is silly, perhaps they had LBA modes as well, I do not remember.

Someone who says LBA is not part of DAE does not know DAE to its full extent, but who am I to judge I have only been doing this for 17 years.

Drive offset and its affect on last sector during CD rip

Reply #22
it seems to be a flaw in EAC's method of always reading (requesting) blocks of sectors upto 64kB per drive read command.  When the rip nears the end of a CD EAC's method of reading can overshoot (past) the last drive readable sector of the CD which will result in an error and then all prior sectors which were readable in that block is dropped by the drive or returned as zeroes.

Thanks for the additional insight on what I thought was already accepted by the OP.

I hope this wasn't an attempt to reaffirm the portion of the discussion that drew my involvement:
It does seem to me that consistency will only be achieved where the drive has an absolute offset of less than or equal to 588. Any offset larger than that will result in a variation subject to EAC settings, offset value and drive capability when ripping the last samples.

Thanks to spoon as well; I appreciate his expertise.  I was clearly not considering the big picture, though I still don't believe I really needed to when addressing the incorrect notion that I quoted above, except to say that my overly-general statement was grossly ill-advised and it should certainly call into question my expertise on the subject.

Regarding the relevancy of hacks, is it still relevant to that drives attribute different data to the frames being returned if they have different offsets?

...or does this issue simply disappear?

Context is everything.