IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Encoding Directly to a Flash based Player, What's the bottleneck here ?
uart
post Apr 10 2005, 17:23
Post #1





Group: Members
Posts: 810
Joined: 23-November 04
Member No.: 18295



I've got a cheap 128MB (usb flash drive type) mp3 player, it's a pretty slow device and it takes about 5 to 6 minutes just to copy approx 120MB of files to it. The other day it occured to me that if I had a fast enough encoder that could just about transcode on the fly at that speed (from lossless stored on the HDD).

Anyway I thought this might be quicker than encoding the files and then copying them, I thought at worst that's how it would be but was hoping that encoding and writing would to some extent be concurrent and I'd save time.

Well the result was surprising, it actually took very much longer than even the sum of the two individual processes (encoding + copying). I was careful to not use intermediate wave file and double checked that the encoder was not trying to place temporary files onto the flash player.

Anyway it took about 11 minutes to encode the selected files and about 5.5 minites to copy them when doing it seperately, but it took about 45 minutes to encode them directly to the flash player. What's the bottleneck here?

PS. At first I was using the "convert" function in foobar but I also tried encoding directly from lame (and fastencc) from wav files using the command line and there was a similar unexpected slow down (as in more than I'd expect due just to the device write speed limitaion) when encoding directly to a flash player.

?any clues?

This post has been edited by uart: Apr 10 2005, 17:36
Go to the top of the page
+Quote Post
NumLOCK
post Apr 10 2005, 19:44
Post #2


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



Hi,

Windows is not efficient in combining write operations. That prevents small repeated writes to flash memory from being efficient.

Example: We have a player which uses a flash memory which is divided in 64kB segments. So, each of the following characters (X, 0, 1) represent:

X = 64kB of undefined data
1 = 64kB of '1' bits
0 = 64kB of '0' bits

Example (ideal):
CODE
data = XXXXXXXX
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10101010


Example (Windows):
CODE
data = XXXXXXXX
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10000000
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10100000
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10101000
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10101010


For every write to flash, a slow ERASE cycle are necessary. That's probably why it's so slow.

Can you try disabling the Windows option "optimize for quick removal" ? It may help.


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post
uart
post Apr 11 2005, 18:48
Post #3





Group: Members
Posts: 810
Joined: 23-November 04
Member No.: 18295



Thanks for the info NumLOCK. So what you're saying is that unless a full segment is written at once then there may be numerous redundant erase cycles on the flash memory.

Is that a correct summary or am I misunderstanding you?
Go to the top of the page
+Quote Post
NumLOCK
post Apr 11 2005, 19:38
Post #4


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



Absolutely well summarized smile.gif

It's still a supposition, but since audio encoders tend to generate one frame at a time, they make many small "file write" calls per megabyte. The hard disks don't care, but on flash memory, it generates many accesses on the same segments.

There are various kinds of flash memory controllers. The most clever ones do "wear leveling", which mean that each time a write is done, they move the target segment around, thus preventing individual segments from dying (as is the case with flash memory).

Most probably, "clever" controllers would combine small writes as well. It all depends on what's in your player.

Other OS's (such as linux) do very agressive write combining in the kernel, so you *might* get the same performance as plain mp3 encoding to hard-disk (or the performance of a plain file copy to flash, whichever is lower).

Regards smile.gif

This post has been edited by NumLOCK: Apr 11 2005, 19:40


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post
ChiGung
post Apr 11 2005, 20:26
Post #5





Group: Members
Posts: 439
Joined: 9-February 05
From: county down
Member No.: 19713



QUOTE (uart @ Apr 10 2005, 05:23 PM)
PS. At first I was using the "convert" function in foobar but I also tried encoding directly from lame (and fastencc) from wav files using the command line and there was a similar unexpected slow down (as in more than I'd expect due just to the device write speed limitaion) when encoding directly to a flash player.

?any clues?

I just gave this a try from the win2000 command prompt , usb 1.1 flash player -there was only a 1 second difference between encoding to the player and the hard disk.
The player shows as a standard removable storage device using disk.sys and write caching is not available in properties (probably managed as a neccessity)

There must be some luck involved then with the players capability and maybe the OS version. This player takes a few minutes to fill up 256 megs.


--------------------
no conscience > no custom
Go to the top of the page
+Quote Post
uart
post Apr 12 2005, 18:04
Post #6





Group: Members
Posts: 810
Joined: 23-November 04
Member No.: 18295



Thanks for the info guys. Based on this I changed the settings (policy) for the USB drive from "optimize for quick removal" to "optimize for performance" and now I can encode directly to the flash drive without any performance penalty. smile.gif

In addition to the "optimize for performance" checkbox I also have an option of "Enable Write Cache". But though I can tick this option every time I return to this properties-policy tab the tick is gone. I suppose it doesn't matter as the "optimize for performance" setting alone seems to be doing the trick.

I'm now very glad I asked this question and got this sorted out, it couldn't be good for the longevity of the flash drive to have all those redundant erase cycles going on.
Go to the top of the page
+Quote Post
NumLOCK
post Apr 12 2005, 20:30
Post #7


Neutrino G-RSA developer


Group: Developer
Posts: 852
Joined: 8-May 02
From: Geneva
Member No.: 2002



Great !! smile.gif

Now, just ensure that you inform windows (using the icon in the notification area) before you unplug your player from the USB port, otherwise the buffers might not get flushed in time.


--------------------
Try Leeloo Chat at http://leeloo.webhop.net
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 28th December 2014 - 12:49