IPB

Welcome Guest ( Log In | Register )

FLAC I/O less efficient than STDIN, Direct file access can be almost twice slower than STDIN
skamp
post Nov 26 2012, 18:26
Post #1





Group: Developer
Posts: 1443
Joined: 4-May 04
From: France
Member No.: 13875



I was optimizing caudec when I came across this oddity. Basically, letting /usr/bin/flac access .flac files on a slowish HDD directly for decoding ('flac -d file.flac') was in one particular case almost twice slower than piping the files to /usr/bin/flac via STDIN ('cat file.flac | flac -d -').

I used a double album for testing, made of 37 tracks for a total of about 1 GiB, located on a HDD that tops out at about 70 MB/s. Incidentally, flac decodes on my machine at a similar rate.

I ran caudec twice (figuratively - I repeated the tests many times) with 8 concurrent processes, for decoding those FLAC files to WAV on a ramdisk. I made sure to drop all caches between each run. First run was with direct file access, and completed in 40 seconds. Second run was with piping to STDIN, and completed in 25 seconds.

The difference was much less pronounced, surprisingly, on a USB flash drive that tops out at 35 MB/s, 34 seconds vs. 30 seconds, and non-existant on a RAID 1 array that tops out at 130 MB/s and on a SSD that tops out at 500 MB/s. I experienced similar differences with WavPack.

Does anyone have any idea of what's going on?


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Axon
post Nov 27 2012, 19:59
Post #2





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



Cool.

Note that --setra and --setra are completely different settings IIRC. Setting these values too high could compromise performance on other applications, so unless the drive is devoted to music, you should probably tune them down appropriately.

I'm rather curious as to if you can improve performance at the default readahead values by instead tuning CFQ params.
Go to the top of the page
+Quote Post
skamp
post Nov 27 2012, 21:59
Post #3





Group: Developer
Posts: 1443
Joined: 4-May 04
From: France
Member No.: 13875



QUOTE (Axon @ Nov 27 2012, 19:59) *
I'm rather curious as to if you can improve performance at the default readahead values by instead tuning CFQ params.


Yes: 23 seconds with CFQ/readahead at 256 (vs. 40 seconds with deadline), 17 seconds with CFQ/readahead at 16384.

I completely forgot that I changed the scheduler to deadline years ago.


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
Axon
post Nov 28 2012, 01:20
Post #4





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



QUOTE (skamp @ Nov 27 2012, 14:59) *
QUOTE (Axon @ Nov 27 2012, 19:59) *
I'm rather curious as to if you can improve performance at the default readahead values by instead tuning CFQ params.


Yes: 23 seconds with CFQ/readahead at 256 (vs. 40 seconds with deadline), 17 seconds with CFQ/readahead at 16384.

I completely forgot that I changed the scheduler to deadline years ago.


huh.gif Yes, I could understand how CFQ would perform better in this sort of workload. But 1MiB readahead still seems a *tad* too high. I was imagining that tweaking things like /sys/block/*/queue/iosched/slice_idle (or other settings described in cfq-iosched.txt) could help.

This post has been edited by Axon: Nov 28 2012, 01:24
Go to the top of the page
+Quote Post

Posts in this topic


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st September 2014 - 05:15