IPB

Welcome Guest ( Log In | Register )

> foobar2000 Tech Support Forum Rules

Please read foobar2000 Tech Support Forum Rules before posting and comply with all the points.
Failure to provide all the information pointed out in the above document in your post is considered wasting other people's time and in extreme cases will lead to your topic getting locked without a reply.


See also: Hydrogenaudio Terms of Service.

2 Pages V   1 2 >  
Reply to this topicStart new topic
Merge FLAC files limited to max 2.00GB?, Trying to merge large flac files
SoleBastard
post Oct 2 2010, 18:40
Post #1





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



I'm trying to merge 2 large FLAC files of about 1.5GB into 1, the resulting FLAC file should be about 3GB. However I get the following error:

CODE
1 out of 2 tracks converted with major problems.

Source: "C:\bigfile1.flac"
Track converted successfully.
Source: "C:\bigfile2.flac"
An error occurred while writing to file (The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters) : "C:\Image.flac"
Additional information:
Encoder stream format: 48000Hz / 2ch / 24bps
Command line: "C:\Foobar2000\flac.exe" -s --ignore-chunk-sizes -8 - -o "Image.flac"
Working folder: C:\
Conversion failed: The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters


The resulting merged image.flac file is cut off at exactly 2.00GB. I've checked both files and they decode fine on their own, and I have the same problem with merging other big flac files too. Hard drive space is sufficient.

As I have many FLAC files >2.00GB its likely not a FLAC limitation but something hardcoded into Foobar2000. Is there any particular reason for this and if not, can this limit please be removed? smile.gif Or does someone have a better idea to merge these files?

(Using Win7 x64, Foobar2000 v1.1, C2D + 8GB RAM, NTFS)

This post has been edited by SoleBastard: Oct 2 2010, 18:43
Go to the top of the page
+Quote Post
Andreasvb
post Oct 2 2010, 18:48
Post #2





Group: Members
Posts: 1337
Joined: 6-March 04
From: Sweden
Member No.: 12509



Seems to be a limitation in Windows.

See this post, http://www.hydrogenaudio.org/forums/index....st&p=483172


--------------------
Windows 8.1u1 Pro x64 Media Center // foobar2000 1.3.3
Go to the top of the page
+Quote Post
SoleBastard
post Oct 2 2010, 18:54
Post #3





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



Well, eac3to has no problem converting DTSHD or TrueHD files into >2GB FLAC files using the reference libFLAC.dll 1.2.1. Thats why I don't think either Windows or FLAC is the problem but rather Foobar2000 itself.

This post has been edited by SoleBastard: Oct 2 2010, 18:56
Go to the top of the page
+Quote Post
Andreasvb
post Oct 2 2010, 18:58
Post #4





Group: Members
Posts: 1337
Joined: 6-March 04
From: Sweden
Member No.: 12509



Is the flac.exe the same version?


--------------------
Windows 8.1u1 Pro x64 Media Center // foobar2000 1.3.3
Go to the top of the page
+Quote Post
SoleBastard
post Oct 2 2010, 19:05
Post #5





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



If you mean if the flac.exe file used by Foobar2000 has the same version as the libFLAC.dll used by eac3to then yes, both are the latest 1.2.1.
Go to the top of the page
+Quote Post
Andreasvb
post Oct 2 2010, 19:24
Post #6





Group: Members
Posts: 1337
Joined: 6-March 04
From: Sweden
Member No.: 12509



What if you try from cmd?
Then you know if it's foobar2000 or not.


--------------------
Windows 8.1u1 Pro x64 Media Center // foobar2000 1.3.3
Go to the top of the page
+Quote Post
SoleBastard
post Oct 2 2010, 19:46
Post #7





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



What do you mean 'from cmd'? I don't know how to do this using flac.exe as I can't seem to find the correct option in 'flac -help'
Go to the top of the page
+Quote Post
lvqcl
post Oct 2 2010, 20:19
Post #8





Group: Developer
Posts: 3336
Joined: 2-December 07
Member No.: 49183



http://www.hydrogenaudio.org/forums/index....st&p=676123
Go to the top of the page
+Quote Post
SoleBastard
post Oct 2 2010, 20:44
Post #9





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



Thanks for the link lvqcl (and I apologize for not finding the thread myself).

I just merged the same files using WavPack and this resulted in a nice 2.25GB file that plays fine. So I guess I was wrong by blaming Foobar2000 but its actually the windows build of FLAC 1.2.1. I guess I'll fire up my Ubuntu boot to try merge the FLACs in there.

However, that still doesn't explain why eac3to can make >2GB FLAC files in Windows without any problems. I'll ask the creator of that program how he manages to do that (see here).

This post has been edited by SoleBastard: Oct 2 2010, 20:56
Go to the top of the page
+Quote Post
Case
post Oct 2 2010, 22:35
Post #10





Group: Developer (Donating)
Posts: 2181
Joined: 19-October 01
From: Finland
Member No.: 322



Here's FLAC 1.2.1 modified to work with file sizes larger than 2 GB. The executable is larger than the official binary because it's statically linked MSVC 2010 compile.

Edit: removed attachment.

This post has been edited by Case: Oct 3 2010, 07:56
Go to the top of the page
+Quote Post
SoleBastard
post Oct 2 2010, 22:55
Post #11





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



Thanks Case, I tried using your flac.exe with Foobar2000 v1.1. However, when the temp file is slightly above 2GB it gives a similar error code as above. To make sure the correct flac.exe is used by Foobar I created a custom convert profile in Foobar and pointed it at your flac.exe. I guess there are some more problems than just the flac.exe file.

As a temporary work-around I merged to big FLAC files into 1 .w64 file with foobar2000. I then converted this .w64 file to flac using eac3to 3.24. Awkward, but it works. smile.gif

This post has been edited by SoleBastard: Oct 2 2010, 23:00
Go to the top of the page
+Quote Post
Squeller
post Oct 3 2010, 00:22
Post #12





Group: Members
Posts: 2351
Joined: 28-August 02
Member No.: 3218



I'm also interested in a solution.

QUOTE (jcoalson @ Dec 30 2009, 03:31) *
libFLAC uses the C stdlib for file i/o. even on my XP box with VS 2005, microsoft's stdlib implementation is still limited to 2 GB (i.e. no 64-bit off_t).

I'm reluctant to add win-specific calls to libFLAC just because MS is intentionally sabotaging portability. every other build of flac works with large files.

(source)
Go to the top of the page
+Quote Post
kode54
post Oct 3 2010, 00:35
Post #13





Group: Admin
Posts: 4579
Joined: 15-December 02
Member No.: 4082



Microsoft's C standard library has extensions for 64-bit file offsets as of some revision from years ago.
Go to the top of the page
+Quote Post
Case
post Oct 3 2010, 08:05
Post #14





Group: Developer (Donating)
Posts: 2181
Joined: 19-October 01
From: Finland
Member No.: 322



Sorry about that. I didn't test piped encoding for larger than 2 GB files. Input file size check broke when "(long)(-1)" got changed to "(__int64)(-1)". Fixed version attached.

The fix involves defining fseeko as _fseeki64 and ftello as _ftelli64 and changing off_t to __int64.

Edit: attachment removed. More off_t to __int64 changes required. Also stat to _stati64 and fstat to _fstat64.

This post has been edited by Case: Oct 3 2010, 09:58
Go to the top of the page
+Quote Post
lvqcl
post Oct 3 2010, 08:52
Post #15





Group: Developer
Posts: 3336
Joined: 2-December 07
Member No.: 49183



Sorry, output FLAC file is still limited by 2,281,813,291 bytes.
Go to the top of the page
+Quote Post
Case
post Oct 3 2010, 10:04
Post #16





Group: Developer (Donating)
Posts: 2181
Joined: 19-October 01
From: Finland
Member No.: 322



It worked for me in a quick test but I should have learned from the first mistake not to be too hasty. If source file was too large the encoded file had no md5 checksum or length info. My original diagnosis of the problem was incorrect.
Now all off_t types are changed to __int64 and remaining 32 bit stuff (stat and fstat) changed to 64 bit versions (_stati64, _fstati64). I can convert 5.61 GB WAV file to ~3 GB FLAC file fine in foobar with pipes but WAV file reader seems to have some issues. Encoding the wav directly from command line reports some incomplete sample error at the end and file lengths don't match in foobar's binary comparator.

Edit: removed attachment.

This post has been edited by Case: Oct 3 2010, 12:47
Go to the top of the page
+Quote Post
SoleBastard
post Oct 3 2010, 10:36
Post #17





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



Sorry, still doesn't work, same non-descriptive error as above.
Go to the top of the page
+Quote Post
lvqcl
post Oct 3 2010, 11:01
Post #18





Group: Developer
Posts: 3336
Joined: 2-December 07
Member No.: 49183



flac - --ignore-chunk-sizes -o a.flac < image.wav =>

-: WARNING: skipping unknown sub-chunk 'ДвЧв' (use --keep-foreign-metadata to keep)
-: ERROR during read while skipping over unsupported sub-chunk

This post has been edited by lvqcl: Oct 3 2010, 11:01
Go to the top of the page
+Quote Post
Case
post Oct 3 2010, 12:49
Post #19





Group: Developer (Donating)
Posts: 2181
Joined: 19-October 01
From: Finland
Member No.: 322



I could dublicate SoleBastard's issue immediately with 24 bit source files and went digging into FLAC's WAV reader. It was designed to only handle 4 GB source files. Attached version no longer has that limit.
Attached File  flac_1.2.1_mod4.zip ( 201.06K ) Number of downloads: 3914
Go to the top of the page
+Quote Post
SoleBastard
post Oct 3 2010, 13:08
Post #20





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



Yay, this one seems to work fine! smile.gif I'll double-check if everything is encoded correctly later on but thanks a lot Case! biggrin.gif
Go to the top of the page
+Quote Post
lvqcl
post Oct 3 2010, 16:59
Post #21





Group: Developer
Posts: 3336
Joined: 2-December 07
Member No.: 49183



QUOTE (SoleBastard @ Oct 2 2010, 23:44) *
I just merged the same files using WavPack and this resulted in a nice 2.25GB file that plays fine.


It's interesting that foobar2000 shows incorrect bitrate of very big wavpack files (> 4 GB?). Winamp and wvunpack -c show the same incorrect bitrate, too.
Go to the top of the page
+Quote Post
SoleBastard
post Oct 4 2010, 19:21
Post #22





Group: Members
Posts: 65
Joined: 13-August 04
Member No.: 16237



FLAC file seems 100%, excellent work Case! biggrin.gif

QUOTE (lvqcl @ Oct 3 2010, 17:59) *
It's interesting that foobar2000 shows incorrect bitrate of very big wavpack files (> 4 GB?). Winamp and wvunpack -c show the same incorrect bitrate, too.


Can confirm here with a 5.3 GB file and the latest WavPack, bitrate is way off (too low). blink.gif
Go to the top of the page
+Quote Post
bryant
post Oct 6 2010, 19:01
Post #23


WavPack Developer


Group: Developer (Donating)
Posts: 1290
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



QUOTE (SoleBastard @ Oct 4 2010, 11:21) *
Can confirm here with a 5.3 GB file and the latest WavPack, bitrate is way off (too low). blink.gif

It's a known problem that WavPack cannot handle files over 4 GB. It was discussed here in detail (and with some workarounds). Note that this a problem with the decoding library only; there is nothing limiting in the file format (and the files are fine).

I basically got this working last summer (along with handling .CAF files) but I still have not gotten a chance to finish it up and release it, but am hopeful for this year.

David
Go to the top of the page
+Quote Post
Squeller
post Oct 7 2010, 17:20
Post #24





Group: Members
Posts: 2351
Joined: 28-August 02
Member No.: 3218



Thanks Case, I'll test it soon. I'll merge some single files albums to flacs>2GB.
Go to the top of the page
+Quote Post
seanyseansean
post Jan 22 2011, 14:14
Post #25





Group: Members (Donating)
Posts: 487
Joined: 12-August 02
From: Cheltenham, UK
Member No.: 3029



QUOTE (Case @ Oct 3 2010, 12:49) *
I could dublicate SoleBastard's issue immediately with 24 bit source files and went digging into FLAC's WAV reader. It was designed to only handle 4 GB source files. Attached version no longer has that limit.
Attached File  flac_1.2.1_mod4.zip ( 201.06K ) Number of downloads: 3914


Thank you, Case. And nice to see an old name around here too!
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 29th July 2014 - 00:11