IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Flac 1.2.1 - Still 2 GB size limit?, flac.exe with fb2k/Converter
Squeller
post Dec 29 2009, 12:34
Post #1





Group: Members
Posts: 2351
Joined: 28-August 02
Member No.: 3218



Hi,

I cannot convert to flac+embedded cue > 2GB.

It's been asked here in 2005, Flac changelog says (for version 1.1.3): "Large file (>2GB) support everywhere". I'm using the latest version 1.2.1 (flac.exe called from fb2k 1.0 Beta5) - Not a file system issue (Saving to an NTFS file system).

FB2K reports:
QUOTE
An error occurred while writing to file (The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters) : "M:\Musik\TEMP\CDImage.flac"
Additional information:
Encoder stream format: 192000Hz / 2ch / 24bps
Command line: "M:\Musik\foobar2000\encoders\flac.exe" -s --ignore-chunk-sizes -6 - -o "CDImage.flac"
Working folder: M:\Musik\TEMP\

File size at that moment is 2.147.494.397 Bytes

This post has been edited by Squeller: Dec 29 2009, 12:35
Go to the top of the page
+Quote Post
GHammer
post Dec 29 2009, 14:47
Post #2





Group: Members
Posts: 224
Joined: 11-May 03
From: China
Member No.: 6546



To isolate the problem, why not call flac from a command prompt instead of from foobar?
If it still fails, look at flac. If it does not fail, look at foobar.
Go to the top of the page
+Quote Post
insane_alien
post Dec 29 2009, 16:04
Post #3





Group: Members
Posts: 64
Joined: 26-July 08
Member No.: 56345



what is the filesystem on M: ?

if its FAT then there's your problem. switch to ntfs.
Go to the top of the page
+Quote Post
Squeller
post Dec 29 2009, 17:06
Post #4





Group: Members
Posts: 2351
Joined: 28-August 02
Member No.: 3218



Isolating a problem is a question of time smile.gif As I wrote, target volume is NTFS. It's all NTFS here. Windows 7 Professional. OK I encoded the 7 GB Wave file via command line. It's 24bit/192khz audio.

QUOTE
c:\Users\Michel\Desktop>m:\Musik\foobar2000\encoders\flac.exe image.wav -s --ignore-chunk-sizes -0 -o image.flac
INFO: Make sure you know what you're doing when using --ignore-chunk-sizes.
Improper use can cause flac to encode non-audio data as audio.
image.wav: WARNING: legacy WAVE file has format type 1 but bits-per-sample=24

Now the encoding process is running... And again, after 2GB (the exact file size is 2.147.484.028 Bytes) have been processed (No, the disk is not full):

QUOTE
image.wav: ERROR during encoding
state = FLAC__STREAM_ENCODER_CLIENT_ERROR

An error occurred while writing; the most common cause is that the disk is full.


This post has been edited by Squeller: Dec 29 2009, 17:21
Go to the top of the page
+Quote Post
Nick.C
post Dec 29 2009, 17:59
Post #5


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Are you sure that it is a WAV file? The WAV structure only accommodates files up to 4,294,967,294 bytes long. Sounds like the file is RIFF64 or BWF.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Squeller
post Dec 29 2009, 18:05
Post #6





Group: Members
Posts: 2351
Joined: 28-August 02
Member No.: 3218



QUOTE (Nick.C @ Dec 29 2009, 17:59) *
Are you sure that it is a WAV file? The WAV structure only accommodates files up to 4,294,967,294 bytes long. Sounds like the file is RIFF64 or BWF.

Hmm, I've already deleted it. I created this wave file only for testing purposes, which made my command line life easier... The original source are multiple, independent flac files, which I have merged to one 7 GB wav file... with fb2k's converter...
BTW I'd also guess there's a file system/block issue here, still maybe on the flac side. The file size we are seeing is very close to 2^31 (2.147.483.648)

This post has been edited by Squeller: Dec 29 2009, 18:28
Go to the top of the page
+Quote Post
jcoalson
post Dec 30 2009, 02:31
Post #7


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



libFLAC uses the C stdlib for file i/o. even on my XP box with VS 2005, microsoft's stdlib implementation is still limited to 2 GB (i.e. no 64-bit off_t).

I'm reluctant to add win-specific calls to libFLAC just because MS is intentionally sabotaging portability. every other build of flac works with large files.
Go to the top of the page
+Quote Post
Squeller
post Dec 30 2009, 08:50
Post #8





Group: Members
Posts: 2351
Joined: 28-August 02
Member No.: 3218



QUOTE (jcoalson @ Dec 30 2009, 02:31) *
I'm reluctant to add win-specific calls to libFLAC just because MS is intentionally sabotaging portability. every other build of flac works with large files.

Regrettably. According to this and this, large file support still seems to be a difficult story.

[fauntleroy]Josh, as we know, you are the most great-hearted flac developer of all times, you'll give us a win32/2gb+ flac anyway, one day, am I right? wink.gif[/fauntleroy]



This post has been edited by Squeller: Dec 30 2009, 08:57
Go to the top of the page
+Quote Post
lvqcl
post Dec 30 2009, 15:18
Post #9





Group: Developer
Posts: 3399
Joined: 2-December 07
Member No.: 49183



Does MinGW has this limitation? If no, maybe libFLAC compiled with MinGW is the solution to this problem.
Go to the top of the page
+Quote Post
chelgrian
post Dec 30 2009, 16:00
Post #10





Group: Members
Posts: 316
Joined: 27-April 03
Member No.: 6228



QUOTE (jcoalson @ Dec 30 2009, 01:31) *
libFLAC uses the C stdlib for file i/o. even on my XP box with VS 2005, microsoft's stdlib implementation is still limited to 2 GB (i.e. no 64-bit off_t).

I'm reluctant to add win-specific calls to libFLAC just because MS is intentionally sabotaging portability. every other build of flac works with large files.


VS2005 and later have _fiseek64 and friends. These take an __int64. fseek is pain anyway because it's generally prototyped to take a long rather than a off_t so you need fseeko, where the size of off_t will depend on the predefines passed to the compiler, or fseeko64 which always takes an off64_t. Even getting it to work on different unicies can be a pain due to confusion over the size of off_t and the presence or lack of o64 versions of functions. I mean just look at all the crap that configure scripts test for an idea of the historical fragmentation of the unix syscall API.

There are a whole host of other reasons to write your own wrappers around CreateFile etc. For example MS cripple the standard C library calls to only deal with paths up to 254 characters long. You have to use CreateFile with a UNICODE escape sequence at the start of the file name to get 32768 characters and it comes with lots of limitations including globbing not working. How much faff to go into rather depends on how how portable you want FLAC to be? Do you need to be able to compile it on ancient versions of Solaris for example?

Basically you end up with compiler detection and preprocessor magic and wrapper functions to sort the mess out whatever you do. For example I have reams of preprocessor magic to deal with the fact that Microsoft refuse to provide support for C99 and all the new types it specifies including fixed width ones. In order to write portable code I have headers which provide all the missing types. I also have stuff to turn off the silly warnings about using "unsafe" API calls. Which is a blatant attempt by MS to get you to use their 'secure' _s replacements which are utterly non-portable.

The other thing you can do is use something like the Apache Portable Runtime to sort the mess out for you however that comes along with the baggage of a piece of code you don't control the copyright and therefore the license of and probably isn't appropriate for FLAC.

Perhaps the answer is to start a project to write a BSD licensed portability layer, I'm certainly game to help if you want to do so.

This post has been edited by chelgrian: Dec 30 2009, 16:01
Go to the top of the page
+Quote Post
chelgrian
post Dec 30 2009, 16:09
Post #11





Group: Members
Posts: 316
Joined: 27-April 03
Member No.: 6228



QUOTE (lvqcl @ Dec 30 2009, 14:18) *
Does MinGW has this limitation? If no, maybe libFLAC compiled with MinGW is the solution to this problem.


MinGW does provide fseeko64 etc. effectively by providing wrappers around MS's C library functions. However firstly FLAC may not compile with MinGW, you have to treat it as another platform as it's a bit of a frankenmix of GNUisms and things understood by the underlying MS libraries and secondly MSVC++ almost certainly produces faster code for Windows than GCC does. However the very latest versions of GCC may have closed the gap.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st October 2014 - 16:40