Welcome Guest ( Log In | Register )

FLAC I/O less efficient than STDIN, Direct file access can be almost twice slower than STDIN
post Nov 26 2012, 18:26
Post #1

Group: Developer
Posts: 1455
Joined: 4-May 04
From: France
Member No.: 13875

I was optimizing caudec when I came across this oddity. Basically, letting /usr/bin/flac access .flac files on a slowish HDD directly for decoding ('flac -d file.flac') was in one particular case almost twice slower than piping the files to /usr/bin/flac via STDIN ('cat file.flac | flac -d -').

I used a double album for testing, made of 37 tracks for a total of about 1 GiB, located on a HDD that tops out at about 70 MB/s. Incidentally, flac decodes on my machine at a similar rate.

I ran caudec twice (figuratively - I repeated the tests many times) with 8 concurrent processes, for decoding those FLAC files to WAV on a ramdisk. I made sure to drop all caches between each run. First run was with direct file access, and completed in 40 seconds. Second run was with piping to STDIN, and completed in 25 seconds.

The difference was much less pronounced, surprisingly, on a USB flash drive that tops out at 35 MB/s, 34 seconds vs. 30 seconds, and non-existant on a RAID 1 array that tops out at 130 MB/s and on a SSD that tops out at 500 MB/s. I experienced similar differences with WavPack.

Does anyone have any idea of what's going on?

See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
Start new topic
post Nov 26 2012, 20:19
Post #2

Group: Members
Posts: 567
Joined: 4-June 02
Member No.: 2220

I would imagine reading and writing to the same mechanical disk would be the culprit. This is supported by the USB drive measurement but that doesn't appear to explain everything. If I am understanding correctly the plain version of decoding is slower than a workaround-type STDIN decoding. Although I am unfamiliar with *nix there are two possibilities (either or both):

OS - handling of STDIO regarding write-buffering via HDD driver and/or CPU;
BIN - when outputting the data from a STDIN source it reads/writes larger chunks and the chunk size works better with the write-buffering and/or CPU cache (I recall something a FB2K regarding something about differences in seek-table when the encoder used STDIN, not sure why this would also affect decoding).

"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post

Posts in this topic

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:


RSS Lo-Fi Version Time is now: 26th November 2015 - 14:47