IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
Flac seeking problems, flac.exe and metaflac.exe difference
Audio_Spyder
post Mar 1 2004, 16:33
Post #1





Group: Members
Posts: 37
Joined: 1-November 02
Member No.: 3675



Hi All

I've been having a few seeking problems with flac files and think some of the problem may be to do with flac.exe

Context:
I rip CDs using EAC in image+cue mode, converting to flac using foobar2k

I had seeking problems in Winamp2 and Musicbox (linux jukebox software). The problem was solved by adding 1 second seekpoints using metaflac.

On new rips, I use foobar2k's cliencoder with flac.exe - for adding seekpoints during the conversion I use the switch:
--seekpoint=1s

The resulting flac files also have quite major seeking problems. On comparing the seek table generated by flac.exe and metaflac.exe, they are very different. If I remove the seektable using metaflac, and then add it back (again using metaflac), the files play with no seeking problems.

The seektable on a typical album is much bigger when using flac.exe. I can't remember how many points it has as I don't have the files in front of me, but it's roughly about 3-4 times as big as the metaflac generated seektable.

Has anyone else come across this or can anyone else confirm this? flac files really need the seekpoints to make them useable, but i don't want to go through a 2 stage process in creating them (mainly because I don't know how to configure foobar2k to do a 2 stage process).
I used to use .ape, but have switched to flac recently as the linux jukebox software mentioned above can't use .ape

Maybe Josh knows if there is a difference in the code between flac and metaflac.

Thanks
-Audio Spyder
Go to the top of the page
+Quote Post
jcoalson
post Mar 1 2004, 17:13
Post #2


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



you're probably on to something. can you do 'metaflac --list' on both files so I can see the difference? theoretically the only difference should be placeholder points in the first file. also, show the exact flac and metaflac command-lines you used to make the two files. thanks.

Josh
Go to the top of the page
+Quote Post
Audio_Spyder
post Mar 1 2004, 18:18
Post #3





Group: Members
Posts: 37
Joined: 1-November 02
Member No.: 3675



sure, I've got the --list output saved to a text file at home. I can try and upload them here and will also let you know the exact commands used in each case.
Go to the top of the page
+Quote Post
jcoalson
post Mar 1 2004, 19:49
Post #4


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



one more thing to note... according to the seeking bug on sourceforge the problem only manifests on FLACs created with the windows flac.exe, not on unix, which may explain why I have never had any seeking problems with FLAC myself.

that gives me another idea... AudioSpyder, do you have a linux box on which you could run 'metaflac --list' on the same two files? if they give different answers then that could point to a bug on the windows side while parsing the metadata.

another thing to note is that the worst thing that could be wrong is a bad seektable and/or bad seek algorithm, both correctable. the underlying audio is fine.

Josh
Go to the top of the page
+Quote Post
Audio_Spyder
post Mar 2 2004, 12:17
Post #5





Group: Members
Posts: 37
Joined: 1-November 02
Member No.: 3675



I've now run a fairly extensive set of tests on windows and linux and it looks like there is a difference between the parsing of metadata.
I'll post a file containg the results here later today.

-Audio Spyder
Go to the top of the page
+Quote Post
Audio_Spyder
post Mar 3 2004, 18:46
Post #6





Group: Members
Posts: 37
Joined: 1-November 02
Member No.: 3675



ok - here are my findings.

-----------------------------------------------------
Background:

Flac files were produced on a WinXP box or Slackware Linux box, with files being stored on a WinXP shared folder

I also used foobar2k v0.8beta8 as a comparison to using the command line tools (flac.exe)

-----------------------------------------------------
File naming convention:

Text files in the (hopefully) attached zip file are named as follows:
[does anyone know how to attach a zip file to a post? I've cut and paste some of the data, but Josh may need to see more]

X_aaa_bbb.txt

X: "W" for files created using metaflac.exe on Windows and "L" using metaflac on Linux
aaa: "win" for Windows versions, "lin" for Linux versions, "foobar" for using foobar2k
bbb: "flac" if the seektable was created using flac, "meta" if it was created using metaflac - I'll come back to the foobar ones later

When encoding using flac.exe or flac, the command line was:
flac test.wav

------------------------------------------------------
Output from the flac when it finished encoding:

Linux:
options: -P 4096 -b 4608 -m -l 8 -q 0 -r 3,3
test.wav: wrote 523020382 bytes, ratio=0.632

Windows:
options: -P 4096 -b 4608 -m -l 8 -q 0 -r 3,3
test.wav: wrote 523060608 bytes, ratio=0.632

The interesting thing here is the different sizes although the input file was the same.

------------------------------------------------------
Command lines used:

when using metaflac to add seekpoints, I used a flac file with no seektable:
metaflac --add-seekpoint=1s filename.flac

Using metaflac to list the metadata block:
metaflac --list filname.flac > X_filename.txt

------------------------------------------------------
Flac files which give a seek error in Winamp:

foobar_flac.flac
foobar_cli_seek.flac
win_flac.flac

i.e. only files created in Windows (but not all of them).

------------------------------------------------------
Results: - since I can't attach the files, I've cut a few lines of the metaflac output.

It looks like windows metadata parsing is doing something different with stream_offset and frame_samples.

Also, there is a difference in the windows flac generated seektable to the linux flac generated seektable

-------------
flac generated seektable

W_win_flac.exe
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=437760, stream_offset=0, frame_samples=829014

L_win_flac.exe
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=437760, stream_offset=829014, frame_samples=4608

W_lin_flac.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=437760, stream_offset=0, frame_samples=829097

L_lin_flac.falc
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=437760, stream_offset=829097, frame_samples=4608


----------------
metaflac generated seektable

W_win_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=41472, stream_offset=0, frame_samples=10086

L_win_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=41472, stream_offset=10086, frame_samples=4608

W_lin_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=41472, stream_offset=0, frame_samples=10086

L_lin_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=41472, stream_offset=10086, frame_samples=4608

------------------------------------------------------
Foobar results:

Foobar seems to do something very strange when encoding flac files.
I tried in 3 ways:
1) use the cliencoder and flac.exe, with default options
- -0 %d%
2) use the cliencoder and flac.exe, with a seektable command line switch
- --seekpoint=1s -o %d%
3) use the FLAC encoder, default options

I hope I've given the correct switches here as I'm writing them down from memory, but the general structure is correct I think

results:
1) creates a file with a seektable
2) creates a file with a very big seektable (much bigger than the equivalent seektable generated by metaflac)
3) creates a file with no seektable

------------------------------------------------------
I hope I've explained things clearly here and it helps nail the seeking problem.

-Audio Spyder

Edited for some formatting problems

This post has been edited by Audio_Spyder: Mar 3 2004, 18:47
Go to the top of the page
+Quote Post
ancl
post Mar 3 2004, 20:44
Post #7





Group: Members (Donating)
Posts: 185
Joined: 29-September 01
Member No.: 54



QUOTE (Audio_Spyder @ Mar 3 2004, 06:46 PM)
[does anyone know how to attach a zip file to a post? I've cut and paste some of the data, but Josh may need to see more]

You can upload a zip-file to the Upload forum, and then link to it from here.
Go to the top of the page
+Quote Post
jcoalson
post Mar 4 2004, 00:58
Post #8


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



hmm, from this it looks like it is either:

1. a bug parsing the seekpoints in libFLAC on windows
2. a bug printing the seekpoints in metaflac on windows

I'm guessing it's #2 since seeking would be totally broken if #1 (because all stream_offsets seem to be 0). if it's #2, it still wouldn't explain the other bug of not being able to seek to samples at n*44100.

I'll step through this tonight.

Josh
Go to the top of the page
+Quote Post
Audio_Spyder
post Mar 4 2004, 10:55
Post #9





Group: Members
Posts: 37
Joined: 1-November 02
Member No.: 3675



QUOTE (ancl @ Mar 3 2004, 07:44 PM)
QUOTE (Audio_Spyder @ Mar 3 2004, 06:46 PM)
[does anyone know how to attach a zip file to a post? I've cut and paste some of the data, but Josh may need to see more]

You can upload a zip-file to the Upload forum, and then link to it from here.

ancl - thanks for the link, exactly what I was looking for.
I've now uploaded the results, follow this link:
zip file of results
Go to the top of the page
+Quote Post
Michael Jordan
post Mar 10 2004, 05:48
Post #10





Group: Banned
Posts: 4
Joined: 10-March 04
Member No.: 12622



Okey
Go to the top of the page
+Quote Post
jcoalson
post Mar 23 2004, 05:42
Post #11


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



OK, I found and fixed two bugs.

the first is only on windows. it explains why the seektable looks wrong with 'metaflac --list'. but the second bug is the real cause of I believe all the seeking problems so far reported (see also bug #851155).

the first bug is that the metaflac code was using printf() w/ %llu for printing out unsigned long long ints (ANSI C), which MSVC6 does not support. it was treating them like unsigned int and screwing up. MSVC6 takes a non-standard %I64u, so I fixed it in CVS.

so the seektables in the FLAC file are fine, and are parsed fine, they just aren't printed properly by metaflac 1.1.0 on windows.

the second bug is much more subtle. what is basically happening is that in some sporadic cases, the byte offset where libFLAC calculated the seek target to be happened to land in front of audio data that looked exactly like a frame header from a future version of FLAC. this is rare since the frame header also has a CRC (I've never run into it myself) but it can happen. the workaround of 1-second seekpoints is consistent with this, since the farther apart seek points are spaced, the more likely for this to happen.

so to the seek routine it looked like it landed in a valid spot in the middle of a FLAC stream it didn't know how to parse, and bailed out. I have fixed it to not do this and that has fixed all the reported cases that I have test data for.

I'm contemplating a bug fix release since I'm still trying to finish up the Ogg FLAC work and it probably shouldn't wait until then.

Josh
Go to the top of the page
+Quote Post
Audio_Spyder
post Mar 23 2004, 12:39
Post #12





Group: Members
Posts: 37
Joined: 1-November 02
Member No.: 3675



Excellent news. Your description of the second bug explains why only some flac files exhibited a seeking problem.
Thanks for looking into this, your efforts are really appreciated.

- Audio Sypder
Go to the top of the page
+Quote Post
kuniklo
post Mar 23 2004, 19:11
Post #13





Group: Developer (Donating)
Posts: 193
Joined: 9-May 02
From: Emeryville, CA
Member No.: 2010



QUOTE (jcoalson @ Mar 23 2004, 04:42 AM)
the second bug is much more subtle.  what is basically happening is that in some sporadic cases, the byte offset where libFLAC calculated the seek target to be happened to land in front of audio data that looked exactly like a frame header from a future version of FLAC.  this is rare since the frame header also has a CRC (I've never run into it myself) but it can happen.  the workaround of 1-second seekpoints is consistent with this, since the farther apart seek points are spaced, the more likely for this to happen.

Thanks for fixing this! I just rebuilt flac from CVS and I can now seek in several files that were previously problematic.
Go to the top of the page
+Quote Post
M
post Mar 23 2004, 19:55
Post #14





Group: Members
Posts: 964
Joined: 29-December 01
Member No.: 830



Josh, I am curious about something. I've always encoded using --best (or -8), and the default 10s seektable interval has always seemed sufficient, since seeking still works - or appears to work - as expected, and I can seek to any point in the file. In his initial post, Kjetil Limkjær said the problem disappears when he uses -8... so is there still a chance I could encounter this bug?

So far, the only time I've ever received an error of any sort from FLAC is when I've tried piping it to SoX. For example, using Speek's Batchenc the following line should downmix a stereo source to mono:
CODE
flac -d -c <infile> | sox  -t raw -r 44100 -s -w -c 1 - <outfile-mono.wav>


But instead, it produces the following message:
CODE
ERROR while decoding data
state = 3:FLAC__STREAM_DECODER_READ_FRAME


The test FLAC in question was encoded from CD-quality (16 bit, 44.1 kHz stereo) audio, using the FLAC 1.1.0 reference encoder.

- M.
Go to the top of the page
+Quote Post
jcoalson
post Mar 23 2004, 20:20
Post #15


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



QUOTE (M @ Mar 23 2004, 01:55 PM)
Josh, I am curious about something. I've always encoded using --best (or -8), and the default 10s seektable interval has always seemed sufficient, since seeking still works - or appears to work - as expected, and I can seek to any point in the file. In his initial post, Kjetil Limkjær said the problem disappears when he uses -8... so is there still a chance I could encounter this bug?

there's still a chance. it will happen any time something that looks like a valid but unparseable frame header (i.e. uses reserved bits) occurs somewhere in the middle of the audio data and a seek lands on it.

QUOTE (M @ Mar 23 2004, 01:55 PM)
So far, the only time I've ever received an error of any sort from FLAC is when I've tried piping it to SoX. For example, using Speek's Batchenc the following line should downmix a stereo source to mono:
CODE
flac -d -c <infile> | sox  -t raw -r 44100 -s -w -c 1 - <outfile-mono.wav>


  But instead, it produces the following message:
CODE
ERROR while decoding data
state = 3:FLAC__STREAM_DECODER_READ_FRAME


  The test FLAC in question was encoded from CD-quality (16 bit, 44.1 kHz stereo) audio, using the FLAC 1.1.0 reference encoder.

does it happen for any FLAC file or just one particular file? I would also expect it to give the same error if you did
CODE
flac -t <infile>


Josh
Go to the top of the page
+Quote Post
M
post Mar 23 2004, 20:34
Post #16





Group: Members
Posts: 964
Joined: 29-December 01
Member No.: 830



QUOTE (jcoalson @ Mar 23 2004, 02:20 PM)
QUOTE (M @ Mar 23 2004, 01:55 PM)
So far, the only time I've ever received an error of any sort from FLAC is when I've tried piping it to SoX. For example, using Speek's Batchenc the following line should downmix a stereo source to mono:
CODE
flac -d -c <infile> | sox  -t raw -r 44100 -s -w -c 1 - <outfile-mono.wav>


  But instead, it produces the following message:
CODE
ERROR while decoding data
state = 3:FLAC__STREAM_DECODER_READ_FRAME


  The test FLAC in question was encoded from CD-quality (16 bit, 44.1 kHz stereo) audio, using the FLAC 1.1.0 reference encoder.

does it happen for any FLAC file or just one particular file? I would also expect it to give the same error if you did
CODE
flac -t <infile>


Josh

This problem seems to be universal, although it only manifests itself in conjunction with SoX. For example, the following line works perfectly well in Batchenc:
CODE
flac -d -c <infile> | lame --alt-preset standard - <outfile.mp3>

I've now tested the FLAC/SoX line with some forty-odd files, all of which came from different sources and were encoded at different times. Same result, every time.

- M.
Go to the top of the page
+Quote Post
Triza
post Apr 16 2004, 02:26
Post #17





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



Josh or other FLAC enthusiasts,

Sorry for resurrecting this post but this is important for me at this point in time.

I read this post over and over and it seems that the encoding of FLAC 1.1.0 was OK. Is this true?

On one hand we have 2 bugs: one which is is just a printout probem of metaflac, another which has something to do with seeking. This latter one implies decoding. So it seems that encoding was fine.

On the other hand Audio_Spyder pointed out that the linux filesize differs from the Windows makes me worry that I misunderstood the thread. It just makes me feel that in the end this was a encoder bug.

I encoded in Windows by EAC with this command line

CODE
-T "ARTIST=%a" -T "ALBUM=%g" -T "DATE=%y" -T "GENRE=%m" -T "TRACKNUMBER=%n" -T "TITLE=%t" -4 --verify -P 65536 -o %d -- %s


and I must admit I never experienced the bug, but I rarely seek. I just want to know if I need to reencode all my music collection now to correct this or indeed encoding was fine. In the latter case could somebody explain why the filesize difference.

Many thanks,

Triza
Go to the top of the page
+Quote Post
Triza
post Apr 16 2004, 15:01
Post #18





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



Hello again,

I do not want to look pedantic here, but please bear with me because the question is to reencode my 250 CD flac correction.

I looked up the related defect in Sourceforge and it seems that files encoded on Linux does not exhibit this problem. Josh said this was a fluke, but mnetown says

QUOTE
Sender: mnewton
Logged In: YES
user_id=546604

interesting...a fluke.  every file i've had difficulty with
after encoding on my windoze box has functioned properly
when encoding with my OSX box.  anyway, just fyi.


So things do not add up to me at least. Also I think the flac files should be identical regardless whether we encode in Linux or Windows unless there is a reference in the flac file that tells the OS type or something

Many thanks for possible answers.

Triza
Go to the top of the page
+Quote Post
jth
post Apr 16 2004, 15:41
Post #19





Group: Members
Posts: 157
Joined: 4-December 01
Member No.: 579



I encoded with flac 1.1.0 on Solaris 9 (UltraSparc), NetBSD 1.6 (i386), and Windows 2000 (i386) and got the same result each time - by checking with metaflac --list on each platform.

The MD5 signature and seektable was the same everywhere.

But you should still run flac -t against your collection if you're still worried.

--jth
Go to the top of the page
+Quote Post
Triza
post Apr 16 2004, 17:29
Post #20





Group: Members
Posts: 367
Joined: 16-November 03
Member No.: 9867



Hi jth,

Thanks. I am in the process to set up a Linux machine as we speak and I am gonna do some tests.

BTW I was worried about the seektable only. I know that the actual music is fine because I encoded with --verify option and also decoded it and compared it against EAC CRC too. I just want to make sure that my collection is in perfect shape.

Triza
Go to the top of the page
+Quote Post
eagleray
post Apr 16 2004, 20:25
Post #21





Group: Members
Posts: 265
Joined: 15-December 03
Member No.: 10452



Several people around here have experienced bugy seeking with flac when playing cue sheet single file rips. MOst changed to Monkey's audio, which is also free. The other alternative is to rip to separate files. I keep asking why anyone would want to compress an entire album to a single file, as opposed to separate files plus a cue sheet, but have not been able to make sense of the answers.
Go to the top of the page
+Quote Post
xmixahlx
post Apr 16 2004, 20:50
Post #22





Group: Members
Posts: 1394
Joined: 20-December 01
From: seattle
Member No.: 693



well, if your player supports cue input, then it doesn't make much of a difference, does it?

...especially if your burner software supports cue/wav --> = perfect replication of CD if done right


later


--------------------
RareWares/Debian :: http://www.rarewares.org/debian.html
Go to the top of the page
+Quote Post
jcoalson
post Apr 16 2004, 21:00
Post #23


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



QUOTE (M @ Mar 23 2004, 02:34 PM)
I've now tested the FLAC/SoX line with some forty-odd files, all of which came from different sources and were encoded at different times. Same result, every time.

ah, maybe sox is closing the pipe after thinking it's got all the wave data. that would be an unrelated problem.

QUOTE (Triza @ Apr 15 2004, 08:26 PM)
I read this post over and over and it seems that the encoding of FLAC 1.1.0 was OK. Is this true?
...
On the other hand  Audio_Spyder pointed out that the linux filesize differs from the Windows makes me worry that I misunderstood the thread. It just makes me feel that in the end this was a encoder bug.


there is no encoding problem. if you read the thread carefully, the difference in filesizes is because he used different ways of adding the seektables in some of the tests.

furthermore, it is not required for a file to encode to the exact same bit pattern on two different platforms. there can be differences because of, for example, different floating point implementations. that doesn't mean there is anything wrong with the encoder or encoding.

Josh
Go to the top of the page
+Quote Post
rtilghman
post Apr 16 2004, 23:03
Post #24





Group: Members
Posts: 56
Joined: 15-March 04
Member No.: 12727



QUOTE (eagleray @ Apr 16 2004, 11:25 AM)
MOst changed to Monkey's audio, which is also free.


I don't understand that one at all. I'm not doubting that it may be true (I have no idea), but switching to Monkey's Audio from FLAC because of this "maybe" seeking problem (that seems, from what I've read, not to be encoder related) seems crazy.

I'm not trying to be defensive of some kind of apostle here, but the analysis that led me to choose FLAC over MA in the first place would make switching to MA unacceptable.

MA's advantages:
- slightly better compression and file sizes (the better compression rarely exceeds 10mb or so, and on a 380mb file 10mb hardly seems like a substantial gain)
- sometimes encodes faster than FLAC (like 12 vs. 13 minutes or something, so not much of an advantage if you plan on ripping an album once)

MA's disadvantages:
- substantially longer decode than FLAC (FLAC is like 100% faster or something)
- more difficult to decode than FLAC (meaning portables that use it, if and when there are any, will have shorter battery life when playing MA files)
- MA doesn't stream as far as I know
- MA has no hardware support, limited player support
- MA isn't open -source (before you counter "but its available to see" it isn't the same thing as open source)

Someone correct me if I'm wrong or missed one of MA's notable strengths somewhere. I just don't understand why MA is still even viable as an option with FLAC on the table, but I'm no expert so maybe I'm missing something.

-rt

This post has been edited by rtilghman: Apr 16 2004, 23:06
Go to the top of the page
+Quote Post
eagleray
post Apr 16 2004, 23:31
Post #25





Group: Members
Posts: 265
Joined: 15-December 03
Member No.: 10452



@rt

I base my statement about Monkey's on the posts (some here and some elsewhere) of several other HA members, at least two of which were developers. Your comparison of flac and MA is correct, as far as I can remember. However, the seeking problem is real when using large full album flac files with cue sheets. With flac files of 17 minutes duration and no cue sheet, I could not find a problem with seeking.

Personally, I don't think that lossless compression makes sense on a portable, even one with a 40 gig drive. Open source is nice, but only if you need to hack the code & know how to do it.

However, after reading some threads on using a single file and a cue sheet for an album, I can't see why some like it, be it Monkey's or flac.

I sometimes use flac myself to do a quick and dirty (burst mode) rip of an album to my hard drive so I can listen to it while leaving my CD/DVD burner free for other things. The P4 compile of flac runs faster on this machine than Monkey's does. Separate files for each track, so seeking is not an issue. These don't stay around too long.

As far as wanting another 10mb of compression on a 380mb file is concerned, making a big deal about small differences is definitely a HA trait.
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 28th December 2014 - 01:50