IPB

Welcome Guest ( Log In | Register )

EAC secure mode test proposal, Testing its real accuracy
tigre
post May 3 2003, 14:41
Post #1


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



Edit : this discussion is splitted from this thread : http://www.hydrogenaudio.org/forums/index....=ST&f=20&t=8849


QUOTE (spoon @ May 3 2003 - 02:23 AM)
QUOTE
Well, actually errors 'always' (99.99...%) return different data for every new read.


How do you know that? it might be 60%, or 10%, because it cannot detect when the data is the same you cannot put a % on it. When AccurateRip makes its way onto EAC then you could put a percentage on it.

A. I haven't read anything about a "validated" percentage like this anywhere (wrong_samples_noticed_by_EAC_due_to_non-matching_results_on_re-read / total_number_of_wrong_samples).


B. This could be tested (volunteers appreciated) like this:

1. You need a (set of) reference .wav file(s). For this either rip a CD in perfect condition, EAC secure mode, ideally compare the CRC to a identical disk's ripped by someone else (cheers to AccurateRip wink.gif ) or choose some .wav files, burn them to CD-R and make sure you keep them unchanged on your HDD.

2. Take the CD(-R), damage it (e.g. different kind of scratches, caused by screwdrivers, sandig paper, ...; dirty fingerprints; draw lines with a thin marker on the data side; ...). Start carefully and try what happens in the next steps. If the disk turns unreadable and causes timing problems/sync errors in burst mode this won't work ...

3. Extract the damaged disk two times in burst mode resulting in a.wav and b.wav.

4. Do wave substractions (Cool Edit: mix paste - overlap - invert checked) r-a.wav=reference.wav - a.wav and r-b.wav = reference.wav - b.wav and a-b.wav = a.wav - b.wav
using Cool Edit or another wave editor capable of that - make sure that dither is disabled in this and the next steps.

5. We want to know how many samples have been turned "wrong" by damaging the disk, so we create a Cool Edit boolean operation like this: if a sample has been read correctly in a.wav, the corresponding sample in r-a.wav will be 0, otherwise something =|= 0. To find samples that are wrong in a OR b, we add r-a.wav and r-b.wav. To prevent errors here we multiply both files with themselves (Cool Edit: mix paste -> modulate), so all values are 0 (correct) or >0 (wrong). To prevent errors here (Cool Edit devides by 32768 after multiplying, so values -182<y<182 will turn 0 due to rounding) we amplify both .wavs by 100dB before. This will lead to 0 values = correct and 32767 values = incorrect (-> amplification by 100 dB makes all samples =|= 0 clip in 16 bit resolution). So the steps are:

5.1. Amplifying: r-a_100.wav = r-a + 100dB; r-b_100.wav = r-a + 100dB
5.2. Copy r-a_100.wav to clipboard and mix paste - modulate r-a_100.wav with data from clipboard, result: r-a_100m.wav; do the same with r-b_100.wav => r-b_100m.wav
5.3. Add (= Cool Edit mix paste - overlap) r-a_100m.wav + r-b_100m.wav = wrong_samples.wav


In the file wrong_samples.wav. we have now all 0 samples stand for correctly extracted samples in both EAC burst mode runs while =|= 0 samples (32767) stand for extraction errors.

6. Now first interesting thing to do IMO would be to have a look at the error postitions. Are there isolated wrong samples or do they occur as blocks / "bursts"?

7. Now we want to know how many of the wrong samples have gotten different values in the two burst mode runs so EAC secure mode would have detected them. For this we
7.1. apply steps 5.1 and 5.2 to a-b.wav resulting in a-b_100m.wav
Step 7.2.
7.2.1. Take a-b_100m.wav and apply Channel Mixer -> Full mix (L = L+R; R = L+R). -> detected_errors.wav
7.2.2.: multiply wrong_samples.wav with detected_errors.wav (Mix Paste Modulate) and substract the result from detected_errors.wav, resulting in the corrected undetected_errors.wav.


The result will contain 0 samples for correct samples and detected errors and 32767 samples for undetected errors.

8. Counting the Number of 32767 samples in wrong_samples.wav (W) and in undetected_errors.wav (O) will enable you to calculate a percentage of correctly detected errors: P=(W-O)/W

Have fun smile.gif

edit:
Additionally we could rip 3 or 4 times in burst mode instead of 2 times to find out if and how often Test & Copy in secure mode without C2 detects addtional errors.

And we could rip in secure mode, C2 enabled (if drive capable) to find out the "P" value for the drive's C2 detection.

edit2: changed some "?" to "=|=" which means "un-eaqual"

edit3:
I missed one thing: If in the 1st 2 reads an error occurs, the sample is re-read in both channels, no matter if the error was detected in one or both channels. So "undetected" errors in one channel are in fact detected if there's a detected error at the same postion on the other channel.

To take this into account in the test, step 7 is modified (see above).

This post has been edited by tigre: May 6 2003, 10:28


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
 
Start new topic
Replies
tigre
post May 6 2003, 16:06
Post #2


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



1st: You ask a question, so I'd appreciate if you answer mine:
QUOTE
QUOTE (DrDoogie @ May 5 2003 - 05:58 AM)

Tigre:
5 sounds good, I'll have a look at it later. But the underlying assumption of your argument, that EAC can be used to test the correctness of a drive's error-reporting / accurate rip, is that EAC doesn't mess up the wav in any way. Which sounds reasonable, only I would like to add that I have seen cases where cdparanoia and plextools would return identical rips, and EAC would disagree. And in this case, that using all variations of ripping modes in EAC (and a number of different speeds for the drive) still gave the exact same disagreement between rips.

So.. perhaps one should test different rippers as well?

It'd be interested what these differences were like. Have you done a wave compare/substraction?


QUOTE (DrDoogie @ May 6 2003 - 05:12 AM)
I think I lost track of the argument somewhere along the way, BUT...

...when you are introducing errors to a CD, some of those errors will be unreported by the drive, and some will be uncorrectable by the drive (and subsequently hidden from the user by muting, interpolation etc. ad nausam).

If the audio extraction program doesn't use C2, there's no way to "report" any errors AFAIK (maybe oversimplified). The drive just passes sample values to the program, which can be corrected/interpolated by the drive of course.

QUOTE
Your fundamental assumptions seem to revolve around trust.
Trust that the drive reports all errors, ...

Nope. In EAC secure mode without C2 that I'm trying to simulate, error detection is done by reading twice and comparing the results. So there's no need for the drive reporting errors.

QUOTE
... trust that errors are handled identically in repetetive reads of the CD ...

I'd rather say I measured (though it would take more measurements with different drives+CDs to validate my findings) that in > 80% of cases errors are not handled identically in repetetive reads. And this is *good* because it makes EAC detect errors, re-read and report errors (=suspicious positions) if < 8 of 80 re-reads are not identical. So far my test has been about error detection, not about correction.

QUOTE
..., and trust that EAC detects a single frame with errors in it.
I found out that EAC does not detect all errors in the 1st two re-reads, so I performed a 2nd part (again: more tests - and details about how EAC works - needed for validation). IMO so far from this no conclusios are possible, only assumptions. Talking about isolated errors: Given the 85% probability of detecting errors by reading twice, it can happen very well, that isolated errors of a few wrong samples are not detected by secure mode - no C2, but would be detected (and corrected) using C2.

QUOTE
But perhaps I have misunderstood.
So... what assumptions are you making in your tests, and what specifically errors (of the drive, of EAC, of...) do they seek to detect?

Your question (something like "How secure is secure mode") made me think about it. I've extracted a lot of CDs so far with EAC and never have been dissapointed, so I trust it, but I'm just curious I think.
So the first question was: EACs error detection (secure mode no C2) works by reading twice and comparing the results. EAC's assumption seemed to be (my assumption) that a sample value returned by a drive either is correct or changing on every re-read, otherwise a wrong value could be returned twice, so the error would be undetected. In my 1st test I wanted to find out if this is true. It is not. Only ~ 85% are detected.

Given EAC's good performance on error detection my next assumption (that has been verified by Pio2001 and posts of Andre Wiedhof on digital in EAC forum) was: EAC re-reads much more than just the detected samples AND (verified by my test:) on (pressed) CDs isolated errors occur very seldom, so the high percentage of detected/corrected errors (tests to find out exact numbers can't be performed until I have detailed information on how EAC works) is caused by detected errors near to undetected that cause the undetected to be included into the re-read+error correction process.

If someone tells me how exactly the re-read process works (see questions in previous post) I'll be able to provide numbers like
"In my test there were ......... wrong samples returned by drive xy in two reads. ... of them would have been undetected taking EACs error detection algorithm into account. This is a percentage of ...".
This is the goal so far for me.

This post has been edited by tigre: May 6 2003, 16:13


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post

Posts in this topic
- tigre   EAC secure mode test proposal   May 3 2003, 14:41
- - Pio2001   For this kind of tests, it would be much better to...   May 3 2003, 15:11
- - mrosscook   tigre, Your test outline is very interesting and ...   May 3 2003, 15:49
- - liekloo   QUOTE (mrosscook @ May 3 2003 - 03:49 PM)Tigr...   May 4 2003, 12:00
- - evereux   QUOTE (mrosscook @ May 3 2003 - 02:49 PM)Ther...   May 4 2003, 12:04
- - Canar   QUOTE (DrDoogie @ May 2 2003 - 07:48 AM)QUOTE...   May 4 2003, 12:16
- - liekloo   QUOTE (evereux @ May 4 2003 - 12:04 PM)QUOTE ...   May 4 2003, 12:35
- - evereux   I have read a file can be manipulated to give the ...   May 4 2003, 12:50
- - Pio2001   EAC's CRC are 32 bits, thus there is one chanc...   May 4 2003, 13:50
- - mrosscook   No, the point I'm trying to make doesn't h...   May 4 2003, 15:05
- - Pio2001   Yes, you're right. From experience, I think t...   May 4 2003, 19:44
- - Pio2001   I think it's sensible to assume that the first...   May 4 2003, 19:48
- - liekloo   QUOTE (Pio2001 @ May 4 2003 - 07:44 PM)Yes, y...   May 4 2003, 20:23
- - Pio2001   I don't think that their secureness is even si...   May 4 2003, 21:04
- - tigre   @All: Thanks for your replies, suggestions and att...   May 5 2003, 09:33
- - tigre   I started a test as proposed: The CD: VA - Stel...   May 5 2003, 09:39
- - DrDoogie   Tigre: 5 sounds good, I'll have a look at it l...   May 5 2003, 14:58
- - tigre   QUOTE (DrDoogie @ May 5 2003 - 05:58 AM)Tigre...   May 5 2003, 15:36
- - tigre   Test part 2 trying to take into account the amount...   May 6 2003, 11:44
- - DrDoogie   I think I lost track of the argument somewhere alo...   May 6 2003, 14:12
- - Pio2001   An error is given by the difference between the re...   May 6 2003, 15:49
- - tigre   1st: You ask a question, so I'd appreciate if ...   May 6 2003, 16:06
- - Pio2001   Tigre, your result is interesting. How does look l...   May 6 2003, 16:20
- - Pio2001   It would be interesting to know how your drive beh...   May 6 2003, 18:41
- - tigre   Thanks to Pio2001 who provided detailed informatio...   May 27 2003, 11:35
- - Pio2001   Wow ! This gives (1-20/10000)*100 = 99.8 % acc...   May 27 2003, 16:30
- - Canar   QUOTE (Pio2001 @ May 27 2003 - 07:30 AM)Wow ...   May 28 2003, 05:42
- - tigre   QUOTE (Canar @ May 27 2003 - 08:42 PM)QUOTE (...   May 28 2003, 08:32
- - tigre   I tested the same scratched CD with another drive:...   May 28 2003, 15:26
- - DrDoogie   QUOTE I'd rather say: We knew before that EAC ...   May 28 2003, 16:00
- - tigre   You're right about swans ... I just havn...   May 28 2003, 16:31
- - Pio2001   Saying EAC is not perfect, and not using it becaus...   May 28 2003, 18:44
- - Pio2001   Tigre, it seems what you call extended reread is a...   Jun 4 2003, 22:50
- - tigre   QUOTE (Pio2001 @ Jun 4 2003 - 01:50 PM)Tigre,...   Jun 5 2003, 00:11
- - Pio2001   I think I'll perform some little tests myself....   Jun 5 2003, 11:13
- - tigre   QUOTE (Pio2001 @ Jun 5 2003 - 02:13 AM)About ...   Jun 5 2003, 11:26
- - DrDoogie   QUOTE (Pio2001 @ May 28 2003 - 09:44 AM)Sayin...   Jun 5 2003, 15:05
- - Pio2001   I agree with you. But the reason for which EAC was...   Jun 5 2003, 19:50
- - kdo   QUOTE (tigre @ Jun 5 2003 - 11:26 AM)- It see...   Jun 7 2003, 21:44
- - Pio2001   QUOTE (kdo @ Jun 7 2003 - 11:44 PM)(Well, exc...   Jun 8 2003, 00:06
- - kdo   QUOTE (Pio2001 @ Jun 8 2003 - 12:06 AM)QUOTE ...   Jun 8 2003, 14:21
- - Pio2001   No doubt, no doubt !   Jun 8 2003, 23:39


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st September 2014 - 16:00