IPB

Welcome Guest ( Log In | Register )

fpMP3Enc: a multi-core MP3 encoder based upon LAME 3.98.2
GeorgeFP
post Aug 2 2009, 12:58
Post #1





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



Hello,

I'd like to introduce my project "fpMP3Enc" - a multicore MP3 encoder based on LAME 3.98.2. It is a sample application/case study for my multicore library "Fiber Pool".

The source code can be downloaded here: http://www.fiberpool.de/en/downloads.html

The current version is still mostly single-threaded for single-file encoding but scales for multiple-files encoding.

The multicore tweaks used in this version are:
- parallel/asynchronous conversion/scaling of PCM data to float samples
- parallel/asynchronous computation of replay gain
- I/O ordering (WAV files are read into memory first before MP3 files are written to disk)

With the following test system
- Intel Q9450@2.66GHz (Quad Core)
- 8 GiB RAM
- Windows Vista x64 (Superfetch disabled)
- 61 WAV files (2.99 GiB, about 5 hours play time)

I've got the following results:
LAME 3.98.2 (x32; rarewares): 24.6x
LAME64 3.98 (x64; mp3tech): 20.8x
fpMP3Enc (x64; single): 23.3x
fpMP3Enc (x64; multi): 80.2x

Single-file encoding is slower than the original LAME because I haven't SSE-optimized the code.
The scale factor in multiple-files encoding is 3.3 compared to LAME and 3.4 compared to "fpMP3Enc single", which is quite good for a first version.

The next steps will be to perform the psycho acoustics computation for each frame in parallel, then the MDCT, and so on.

Compile and usage:
- You need Visual Studio 2008 (and obviously Windows) to compile the project.
- SSE4 is enabled by default. To disable: #undef USE_SSE4
- Memory control is disabled by default in this version. This can lead to pagefile swapping if too many files are to be encoded. You can enable memory control by using a different 'MEMORY_MODE' macro (experimental).
- Win32 version: The total WAV file size sum must not exceed 400MiB.
- Win64 version: The total WAV file size sum must not exceed 400GiB.
- Input: WAV 16-bit stereo, 44.1kHz
- Output: MP3 Stereo/Joint-Stereo, 44.1kHz

I'd like to know what you think about this project so contact me if you have any comments or feedback.
George
Go to the top of the page
+Quote Post
 
Start new topic
Replies
nazgulord
post Aug 2 2009, 15:55
Post #2





Group: Members
Posts: 43
Joined: 15-November 05
Member No.: 25786



I'm not sure, but isn't it that foobar2000 just starts 2 instances of the encoder rather than one multithreaded instance? Out of curiosity, would the multithreaded one be better?
Go to the top of the page
+Quote Post
saratoga
post Aug 2 2009, 17:21
Post #3





Group: Members
Posts: 5054
Joined: 2-September 02
Member No.: 3264



QUOTE (nazgulord @ Aug 2 2009, 10:55) *
Out of curiosity, would the multithreaded one be better?


If you only have 1 file, then yes. Otherwise, running two encodes in parallel will almost certainly be faster due to less overhead from threading, synchronization, etc.
Go to the top of the page
+Quote Post
Fandango
post Aug 2 2009, 17:23
Post #4





Group: Members
Posts: 1549
Joined: 13-August 03
Member No.: 8353



QUOTE (Mike Giacomelli @ Aug 2 2009, 18:21) *
Otherwise, running two encodes in parallel will almost certainly be faster due to less overhead from threading, synchronization, etc.
Are you sure?
Go to the top of the page
+Quote Post

Posts in this topic
- GeorgeFP   fpMP3Enc: a multi-core MP3 encoder based upon LAME 3.98.2   Aug 2 2009, 12:58
- - Fandango   QUOTE (GeorgeFP @ Aug 2 2009, 13:58) I...   Aug 2 2009, 14:51
- - Kitsuned   Doesn't foobar2000 already do this if you have...   Aug 2 2009, 15:49
- - nazgulord   I'm not sure, but isn't it that foobar2000...   Aug 2 2009, 15:55
|- - Mike Giacomelli   QUOTE (nazgulord @ Aug 2 2009, 10:55) Out...   Aug 2 2009, 17:21
|- - Fandango   QUOTE (Mike Giacomelli @ Aug 2 2009, 18:2...   Aug 2 2009, 17:23
|- - Mike Giacomelli   QUOTE (Fandango @ Aug 2 2009, 12:23) QUOT...   Aug 2 2009, 17:38
|- - GeorgeFP   QUOTE (Kitsuned @ Aug 2 2009, 16:49) Does...   Aug 2 2009, 19:18
|- - Mike Giacomelli   QUOTE (GeorgeFP @ Aug 2 2009, 14:18) QUOT...   Aug 2 2009, 19:23
||- - GeorgeFP   QUOTE (Mike Giacomelli @ Aug 2 2009, 20:2...   Aug 2 2009, 20:04
|- - q_b6   QUOTE (GeorgeFP @ Aug 3 2009, 02:18) QUOT...   Jan 24 2010, 02:12
|- - GeorgeFP   QUOTE (q_b6 @ Jan 24 2010, 02:12) I think...   Jan 24 2010, 19:11
- - Fandango   foobar2000 does not use multi-threading encoders, ...   Aug 2 2009, 16:56
- - Mike Giacomelli   So you're getting a huge speed up by essential...   Aug 2 2009, 20:40
|- - GeorgeFP   QUOTE (Mike Giacomelli @ Aug 2 2009, 21:4...   Aug 2 2009, 21:28
- - Axon   Yeah, that was probably a bad example. In the mo...   Aug 2 2009, 21:15
|- - GeorgeFP   QUOTE (Axon @ Aug 2 2009, 22:15) In the m...   Aug 2 2009, 22:13
|- - Fandango   QUOTE (GeorgeFP @ Aug 2 2009, 23:13) In ...   Aug 2 2009, 22:50
|- - GeorgeFP   QUOTE (Fandango @ Aug 2 2009, 23:50) Will...   Aug 3 2009, 07:00
|- - GeorgeFP   Hi again, I've updated the application and ad...   Aug 5 2009, 19:53
- - GeorgeFP   Hello, it's me again... The last three weeks ...   Aug 26 2009, 22:03
|- - Alexxander   QUOTE (GeorgeFP @ Aug 26 2009, 23:03) The...   Aug 27 2009, 08:41
|- - GeorgeFP   QUOTE (Alexxander @ Aug 27 2009, 09:41) H...   Aug 27 2009, 10:12
- - spoon   x80 encoding speed equates to 14MB a second read (...   Aug 26 2009, 23:09
|- - GeorgeFP   QUOTE (spoon @ Aug 27 2009, 00:09) x80 en...   Aug 27 2009, 09:45
|- - GeorgeFP   Hello, the "final" version of fpMP3Enc ...   Sep 30 2009, 07:04
- - hlloyge   I am sorry, but where are compiled binaries? I don...   Sep 30 2009, 07:48
|- - GeorgeFP   QUOTE (hlloyge @ Sep 30 2009, 08:48) I am...   Sep 30 2009, 08:44
|- - PatchWorKs   QUOTE (GeorgeFP @ Sep 30 2009, 09:44) The...   Sep 30 2009, 10:51
|- - skamp   QUOTE (PatchWorKs @ Sep 30 2009, 11:51) W...   Sep 30 2009, 13:15
|- - GeorgeFP   QUOTE (PatchWorKs @ Sep 30 2009, 11:51) W...   Sep 30 2009, 13:31
- - twist3d   Any chance getting a compile to rarewares?   Oct 2 2009, 12:27
- - GeorgeFP   Hi again, meanwhile, I've added ID3 tagging s...   Oct 28 2009, 10:02
- - punkrockdude   Anyone who has compiled this encoder that could se...   Oct 28 2009, 11:43
|- - GeorgeFP   QUOTE (punkrockdude @ Oct 28 2009, 12:43)...   Oct 28 2009, 20:02
|- - GeorgeFP   QUOTE (GeorgeFP @ Oct 28 2009, 21:02) I a...   Oct 29 2009, 10:53
|- - jamesbaud   QUOTE (GeorgeFP @ Oct 29 2009, 01:53) QUO...   Dec 3 2009, 08:29
- - GeorgeFP   The download link is: http://www.rarewares.org/mp3...   Nov 3 2009, 06:39
- - Cokemonkey11   Any chance in a GUI frontend? I'm interested i...   Nov 3 2009, 07:08
|- - GeorgeFP   QUOTE (Cokemonkey11 @ Nov 3 2009, 08:08) ...   Nov 3 2009, 07:46
||- - Cokemonkey11   QUOTE (GeorgeFP @ Nov 2 2009, 23:46) QUOT...   Nov 6 2009, 02:45
||- - GeorgeFP   QUOTE (Cokemonkey11 @ Nov 6 2009, 03:45) ...   Nov 6 2009, 10:30
|- - punkrockdude   QUOTE (Cokemonkey11 @ Nov 3 2009, 08:08) ...   Nov 3 2009, 08:10
- - sld   The figures thrown around here are fantastic. Many...   Nov 3 2009, 07:43
- - Brent   Have you thought about using .net/mono with gtk#? ...   Nov 6 2009, 13:32
|- - Xire   Any plans to support STDIN/STDOUT?   Nov 6 2009, 13:58
||- - GeorgeFP   QUOTE (Xire @ Nov 6 2009, 14:58) Any plan...   Nov 6 2009, 15:08
|- - GeorgeFP   QUOTE (Brent @ Nov 6 2009, 14:32) Have yo...   Nov 6 2009, 14:57
- - hlloyge   C2Q, 32 bit encoder, 32 bit windows 7, i know it s...   Nov 6 2009, 17:58
|- - GeorgeFP   QUOTE (hlloyge @ Nov 6 2009, 17:58) C2Q, ...   Nov 6 2009, 18:22
||- - hlloyge   QUOTE (GeorgeFP @ Nov 6 2009, 19:22) From...   Nov 6 2009, 22:11
|- - Alexxander   QUOTE (hlloyge @ Nov 6 2009, 17:58) It wo...   Nov 6 2009, 18:58
|- - GeorgeFP   QUOTE (Alexxander @ Nov 6 2009, 18:58) Ex...   Nov 6 2009, 20:57
- - smeargol   Very good work! I would like to try it out, bu...   Nov 10 2009, 00:34
|- - GeorgeFP   QUOTE (smeargol @ Nov 10 2009, 00:34) So,...   Nov 10 2009, 07:08
|- - smeargol   QUOTE (GeorgeFP @ Nov 10 2009, 08:08) I m...   Nov 10 2009, 14:01
|- - GeorgeFP   QUOTE (smeargol @ Nov 10 2009, 14:01) It ...   Nov 18 2009, 18:01
|- - smeargol   QUOTE (GeorgeFP @ Nov 18 2009, 19:01) Bad...   Nov 20 2009, 22:58
- - sandhuatha   Thanks a lot for this utility. I am very happy to ...   Dec 5 2009, 21:59
|- - GeorgeFP   QUOTE (sandhuatha @ Dec 5 2009, 21:59) Ho...   Dec 5 2009, 23:51
- - edwardar   Any chance someone could give a foobar2000 command...   Dec 30 2009, 17:16
|- - GeorgeFP   QUOTE (edwardar @ Dec 30 2009, 17:16) Any...   Dec 31 2009, 10:08
- - -sanb-   how can i use them with winamp?   Jan 2 2010, 15:06
- - flapane   Will you add mp3 transcoding (mp3 to mp3) support?   Jan 18 2010, 22:00
|- - GeorgeFP   QUOTE (flapane @ Jan 18 2010, 22:00) Will...   Jan 18 2010, 22:41
- - LordCorvin   Sorry if that've been asked already, but is th...   Jan 23 2010, 20:06
|- - GeorgeFP   QUOTE (LordCorvin @ Jan 23 2010, 20:06) S...   Jan 23 2010, 20:46
|- - LordCorvin   QUOTE (GeorgeFP @ Jan 23 2010, 20:46) QUO...   Jan 23 2010, 22:27
|- - GeorgeFP   QUOTE (LordCorvin @ Jan 23 2010, 22:27) S...   Jan 23 2010, 23:04
- - flapane   Hi, could you explain me why it runs single thread...   Jun 18 2011, 17:55
- - GeorgeFP   By design, the encoder will run with at least thre...   Jun 18 2011, 20:14
- - JJZolx   Any thoughts to updating this to make it based on ...   May 21 2012, 02:10
|- - GeorgeFP   QUOTE (JJZolx @ May 21 2012, 03:10) Any t...   May 21 2012, 19:17
|- - lar1r   QUOTE (GeorgeFP @ May 21 2012, 19:17) QUO...   Aug 10 2012, 21:48
|- - GeorgeFP   QUOTE (lar1r @ Aug 10 2012, 22:48) Intere...   Aug 11 2012, 11:33
|- - lar1r   QUOTE (GeorgeFP @ Aug 11 2012, 11:33) QUO...   Aug 13 2012, 15:24
- - JJZolx   George, has there been any progress on an update t...   Jan 17 2014, 16:25
- - goa pride   George, has there been any progress on an update t...   Jan 17 2014, 22:48


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 31st October 2014 - 10:04