IPB

Welcome Guest ( Log In | Register )

2 Pages V  < 1 2  
Reply to this topicStart new topic
flac behind the scenes with EAC..., flac, EAC, and queued compression...
Triza
post Dec 27 2003, 02:36
Post #26





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



QUOTE
I would expect that if verify failed, then flac -t would also fail, unless the verify decoder failed because of a mismatch of decoded audio data. every time I had a report like that it turned out to be a hardware problem.


Thanks Josh for getting back to me on this.

So based on this quote, it is POSSIBLE that --verify failed because of a mismatch, but flac -t would not be able to detect it.

However small of the probability of this, I decided to check this. As I said I had a brief encounter with a HW problem, which I think I eradicated, but I want to be on my guard, and it is possible that a HW problem would just instigate a mismatch. BTW I love FLAC even more by the fact that it help me to find my HW issue, which layed dormant or elusive for weeks.

For those who are in the same boat as I am and what to check their archives retrospective:

I suggested earlier to use CUE sheet to find out the playtime and compare it with the decoded flac file to find out if we had a --verify faliure. Well, this was a wrong idea because the CUE sheet does not say how long a track should last. Sadly the logfile does not contain that either. Hoewever the log file has the CRC and I saved the logs, so today I was frantically tried to inverse engineer EAC CRC calculation. The idea is to decode the flac file and generate the CRC and compare it against the EAC log. The whole task was exacerbated by the fact that I set my EAC so that it skips silence when CRC is calculated. It took me for a while to find out what does this mean exactly. For your benefit here is a quickly knocked-up code that calculates the CRC. The code reads the file on STDIN and writes the CRC onto STDOUT. If there is an error or something amiss it asserts. There are no bells and whistles and purpose built for my checking procedure and I need this only temporarily. If you set EAC no to skip silence when CRC is calculated, just look into the code. There is a comment that tells you how to change the code to calculate CRC that way. This compiles on Cygwin. I hope you may find it useful. BTW the CRC calculation is from cksfv GNU software:

CODE
#include <fcntl.h>
#include <unistd.h>
#include <sys/types.h>
#include <assert.h>
#include <stdio.h>


#define BUFFERSIZE 65536   /* (64k) buffer size for reading from the file */

static const unsigned long crctable[256] = {
 0x00000000, 0x77073096, 0xee0e612c, 0x990951ba,
 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec,
 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940,
 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116,
 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a,
 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818,
 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e,
 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c,
 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0,
 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086,
 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4,
 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a,
 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe,
 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc,
 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252,
 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60,
 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04,
 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a,
 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38,
 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e,
 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2,
 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0,
 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6,
 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94,
 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d,
};


void crc32(register int fd, unsigned long *main_val)
{
 char                  buf[BUFFERSIZE], *p;
 int                   len = 0, nr;
 unsigned long         crc = ~0, crc32_total = ~0;
    
 while ((nr = read(fd, buf, sizeof(buf))) > 0){
   assert(nr % 2 == 0);
   for (len += nr, p = buf; nr--; ++p) {
     // Skip silence when calculate CRC. If EAC is set so that it
     // does not skip silence when CRC is calculated, remove this if
     // statement, or change its condition to non-zero, so it is always
     // true.
     if(*((__int16_t*)p) != 0){
       crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
       crc32_total = (crc >> 8) ^ crctable[(crc32_total ^ *p) & 0xff];

       nr--;
       ++p;

       crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
       crc32_total = (crc >> 8) ^ crctable[(crc32_total ^ *p) & 0xff];
     }
     else{
       nr--;
       ++p;
     }    
   }
 }

 assert(nr >= 0);    

 *main_val = ~crc;
}


int main()
  {
  unsigned long Crc;

  // Skip the WAV header.
  const int Ret = lseek(STDIN_FILENO, 44, SEEK_SET);
  assert(Ret >= 0);

  crc32(STDIN_FILENO, &Crc);

  printf("%0X\n", Crc);
  }
Go to the top of the page
+Quote Post
Triza
post Dec 27 2003, 02:43
Post #27





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



QUOTE (dewey1973 @ Dec 26 2003, 07:57 AM)
QUOTE (Triza @ Dec 25 2003, 10:08 AM)
The manual says that it aborts IMMEDIATELY if there is a mismatch when
--verify option is used. Does this mean that that flac file would be
CORRUPT, that is, it would not be closed down properly (e.g. MD5
signature is not generated etc) if this --verify fails.

One failed on me before. If I remember correctly there was just no flac file created. That's why I always check the album to make sure all the tracks are there.

Dear dewey1973,

Josh was explicit in this thread about the fact that flac file could be left behind, so your checking for the existence of flac file is incorrect.
Go to the top of the page
+Quote Post
Triza
post Dec 27 2003, 03:17
Post #28





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



QUOTE (jcoalson @ Dec 26 2003, 10:40 AM)
QUOTE (phwip @ Dec 26 2003, 05:08 AM)
QUOTE (phwip @ Dec 11 2003, 12:17 PM)
HOWEVER: I would still suggest that the change to FLAC to create a different file name (eg. SongTitle.VERIFYFAILED.flac) on verification failure would be a worthwhile addition.  Even if this issue does get resolved with EAC then it would still be a problem for any other frontend for FLAC that does not take into account the return code.

Josh, are you dead set against this idea?

no, it's actually growing on me. will probably be in the next release, whenever that is.

Josh

Hi folks,

Josh, please do not do that!

Being a engineer myself, who writes a lot of code for work, I must say I have to disagree with this proposal. 1stly FLAC does everything correctly and this would be an unnecessary change, however small it is. This is a clear cut EAC problem so I think we should keep asking EAC developer for a fix. Why I do not like this?
Partly because every code change could introduce bugs inside FLAC and outside FLAC. It seems Josh SW work pretty defensive so internal bug is unlikely. However I and god knows how many thousands of others write sw like EAC or scripts using FLAC and the last thing we want is that if we tell FLAC that the output file should be called x, which you can do with the command line params, then if --verify fails it becomes x.VERIFYFAILED. I am in the SW business day in day out so I've seen a lot of bugs emerges from small changes like this in SW that uses other SW whose behaviour changed (FLAC in our case). You may find this unlikely, but there are thousands of folks write millions of code. I may already have x.VERIFYFAILED file. Or that filename is so long that it is not supported by my OS or storage medium used during the operation. Or the sudden emerge of an unexpected file causes a meltdown in the SW that uses FLAC. Considering that FLAC does handle this case adequatly and which is documented clearly so assumed by everybody I find this change unnecessary, which could break backward compatibility with FLAC 1.1.0.

Josh, if you consider this change, I for one hope you will not, then please add a new command line switch to enable this behavior, so the current users will not see behavior change.

I think the best solution is a wrapper function around the external compressor (whatever it is) which logs non-zero exit codes. I'll look into this tomorrow. It sould be a doddle.

Triza
Go to the top of the page
+Quote Post
phwip
post Dec 27 2003, 09:31
Post #29





Group: Members (Donating)
Posts: 345
Joined: 26-February 03
From: Norwich, UK
Member No.: 5189



QUOTE (jcoalson @ Dec 26 2003, 06:40 PM)
QUOTE (phwip @ Dec 26 2003, 05:08 AM)
QUOTE (phwip @ Dec 11 2003, 12:17 PM)
HOWEVER: I would still suggest that the change to FLAC to create a different file name (eg. SongTitle.VERIFYFAILED.flac) on verification failure would be a worthwhile addition.  Even if this issue does get resolved with EAC then it would still be a problem for any other frontend for FLAC that does not take into account the return code.

Josh, are you dead set against this idea?

no, it's actually growing on me. will probably be in the next release, whenever that is.

Josh

Excellent. Thank you.
Go to the top of the page
+Quote Post
jcoalson
post Dec 27 2003, 20:30
Post #30


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



QUOTE (Triza @ Dec 26 2003, 08:36 PM)
I suggested earlier to use CUE sheet to find out the playtime and compare it with the decoded flac file to find out if we had a --verify faliure. Well, this was a wrong idea because the CUE sheet does not say how long a track should last.

The FLAC cuesheet always contains the lead-out location as a special track. If you trust the length of the original WAVE file you can compare it against the leadout in the FLAC cuesheet, which is computed from the WAVE headers and written before audio encoding starts (so it will be there even if encoding stops with an error).

QUOTE (Triza @ Dec 26 2003, 09:17 PM)
Josh, if you consider this change, I for one hope you will not, then please add a new command line switch to enable this behavior, so the current users will not see behavior change.

Quite right, it would be an option.

Josh
Go to the top of the page
+Quote Post
Iconoclast_a
post Dec 27 2003, 21:44
Post #31





Group: Members
Posts: 69
Joined: 6-June 03
Member No.: 7038



The issue about possible problems with other programs that use FLAC could be avoided by, instead of changing the filename, writing an error logfile to the same directory in case of any mistakes during the verification process.

So, instead of having this: "SongTitle.VERIFYFAILED.flac"

We would have "SongTitle.flac" & "SongTitle.VERIFYFAILED.log"

This post has been edited by Iconoclast_a: Dec 27 2003, 21:45
Go to the top of the page
+Quote Post
Triza
post Dec 27 2003, 23:47
Post #32





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



QUOTE (Josh)
The FLAC cuesheet always contains the lead-out location as a special track.  If you trust the length of the original WAVE file you can compare it against the leadout in the FLAC cuesheet, which is computed from the WAVE headers and written before audio encoding starts (so it will be there even if encoding stops with an error).


I must admit that I am lost.

I do not use FLAC CUE sheets. I use the one EAC generated and I saved them separately. (This is because I want to tag each track and therefore each track has to be a separate file.) I use the so-called "non-compliant" cue sheet when the pregaps are appended to the previous track. Here is an example

CODE
PERFORMER "Quimby"
TITLE "Ékszerelmére"
FILE "Quimby - Ékszerelmére\01 - Androidő.wav" WAVE
 TRACK 01 AUDIO
   TITLE "Androidő"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 01 00:00:00
 TRACK 02 AUDIO
   TITLE "Pedofíling"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 03:59:19
FILE "Quimby - Ékszerelmére\02 - Pedofíling.wav" WAVE
   INDEX 01 00:00:00
 TRACK 03 AUDIO
   TITLE "Halleluja"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 03:19:39
FILE "Quimby - Ékszerelmére\03 - Halleluja.wav" WAVE
   INDEX 01 00:00:00
 TRACK 04 AUDIO
   TITLE "Libidó"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 04:38:44
FILE "Quimby - Ékszerelmére\04 - Libidó.wav" WAVE
   INDEX 01 00:00:00
 TRACK 05 AUDIO
   TITLE "Olé"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 05:15:64
FILE "Quimby - Ékszerelmére\05 - Olé.wav" WAVE
   INDEX 01 00:00:00
 TRACK 06 AUDIO
   TITLE "Az ördög magyar hangja"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 04:40:02
FILE "Quimby - Ékszerelmére\06 - Az ördög magyar hangja.wav" WAVE
   INDEX 01 00:00:00
 TRACK 07 AUDIO
   TITLE "Hintalógalopp"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 05:33:02
FILE "Quimby - Ékszerelmére\07 - Hintalógalopp.wav" WAVE
   INDEX 01 00:00:00
 TRACK 08 AUDIO
   TITLE "Az otthontalanság otthona"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 04:06:12
FILE "Quimby - Ékszerelmére\08 - Az otthontalanság otthona.wav" WAVE
   INDEX 01 00:00:00
 TRACK 09 AUDIO
   TITLE "Mennyből az angyal"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 06:00:46
FILE "Quimby - Ékszerelmére\09 - Mennyből az angyal.wav" WAVE
   INDEX 01 00:00:00
 TRACK 10 AUDIO
   TITLE "Álmatlan dal"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 04:23:71
FILE "Quimby - Ékszerelmére\10 - Álmatlan dal.wav" WAVE
   INDEX 01 00:00:00
 TRACK 11 AUDIO
   TITLE "Unom"
   PERFORMER "Quimby"
   ISRC HUmOOO990029
   INDEX 00 04:50:43
FILE "Quimby - Ékszerelmére\11 - Unom.wav" WAVE
   INDEX 01 00:00:00


I fail to see how can I figure out the length of each wav file. Could it be that I miss something or what you say is only applicable to FLAC CUE sheets? Note that INDEX 00 is not always present. It is here, but as we know INDEX 00 is just the start of the pregap as opposed to the end of the track.

Anyway as it stands now I am going to use CRC from log file I also saved.

QUOTE (Josh)
Quite right, it would be an option.

Good man.

Triza
Go to the top of the page
+Quote Post
jcoalson
post Dec 28 2003, 01:45
Post #33


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



QUOTE (Triza @ Dec 27 2003, 05:47 PM)
QUOTE (Josh)
The FLAC cuesheet always contains the lead-out location as a special track.  If you trust the length of the original WAVE file you can compare it against the leadout in the FLAC cuesheet, which is computed from the WAVE headers and written before audio encoding starts (so it will be there even if encoding stops with an error).


I must admit that I am lost.

I do not use FLAC CUE sheets. I use the one EAC generated and I saved them separately. (This is because I want to tag each track and therefore each track has to be a separate file.)

OK, it won't help things; it's only possible if you rip a whole CD to a single FLAC and add the cuesheet while encoding with flac --cuesheet. the FLAC internal cuesheet has a little more info in it.

Josh
Go to the top of the page
+Quote Post
Triza
post Dec 28 2003, 03:12
Post #34





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



As I promised:

http://www.hydrogenaudio.org/show.php/showtopic/16855
Go to the top of the page
+Quote Post
phwip
post Jan 2 2004, 00:13
Post #35





Group: Members (Donating)
Posts: 345
Joined: 26-February 03
From: Norwich, UK
Member No.: 5189



QUOTE (Iconoclast_a @ Dec 27 2003, 08:44 PM)
The issue about possible problems with other programs that use FLAC could be avoided by, instead of changing the filename, writing an error logfile to the same directory in case of any mistakes during the verification process.

So, instead of having this: "SongTitle.VERIFYFAILED.flac"

We would have "SongTitle.flac" & "SongTitle.VERIFYFAILED.log"

Sounds like a good idea to me.
Go to the top of the page
+Quote Post
phwip
post Jan 2 2004, 23:01
Post #36





Group: Members (Donating)
Posts: 345
Joined: 26-February 03
From: Norwich, UK
Member No.: 5189



QUOTE (phwip @ Dec 11 2003, 12:17 PM)
I have suggested that perhaps there could be a box in the compression options where the user could enter the encoder's successful return code (eg. zero for FLAC).  If they do so and then the encoder returns a different value then EAC should report an error.

Excellent news: Andre has implemented this. It will be in the next release of EAC.
Go to the top of the page
+Quote Post

2 Pages V  < 1 2
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd August 2014 - 09:38