IPB

Welcome Guest ( Log In | Register )

5 Pages V  < 1 2 3 4 5 >  
Reply to this topicStart new topic
A multicore FLAC encoder
GeorgeFP
post Dec 4 2009, 22:02
Post #51





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (Chinch @ Dec 2 2009, 06:25) *
1) the encoding algorithm is standardized, correct? that is, obviously it is going to provide lossless encoding/decoding, and compress to (at least) the same filesize as one encoded with the original FLAC.exe? i know there were some projects going on trying to increase performance/compression on this codec.... which i don't like, standards are made for reasons wink.gif

The encoding algorithm is exactly the same (except some SSE optimizations) as the original FLAC.

QUOTE
2) you say the current algorithm for encoding is i/o bound, i'm assuming this means the encoder can encode faster than the hard drive can read/supply the data, is this what you mean? unfortunately, this probably means that on faster systems, the speed improvement (vs. the original encoder) will have a tendency to decline the faster the system gets. fortunately, this also will improve with current and future tech like SSD drives... wink.gif

The encoder is embarassingly parallel so sooner or later the I/O will become the bottleneck.
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 4 2009, 22:21
Post #52





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (greynol @ Dec 4 2009, 18:12) *
Considering that EAC only passes one file at a time and has its own internal ability to make use of multiple processes, can someone please explain how you derive a benefit from this program with EAC?

If it's possible to utilize all cores with one single file, why would you want to process more than one file in parallel?
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 22:29
Post #53





Group: Super Moderator
Posts: 10044
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Perhaps you could have made it a little more clear as to what your encoder does so that idiots like me wouldn't have to ask such stupid questions. wink.gif


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 4 2009, 22:43
Post #54





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (greynol @ Dec 4 2009, 22:29) *
Perhaps you could have made it a little more clear as to what your encoder does so that idiots like me wouldn't have to ask such stupid questions. wink.gif

So, you did NOT visit my web site? Take a look here.

BTW, it seems I've found the bug I've mentioned before - one hour full-speed encoding without a hang.
The new version is online, the names are the same.
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 22:50
Post #55





Group: Super Moderator
Posts: 10044
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Well no. Why didn't you provide that link in the first place?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
jamesbaud
post Dec 5 2009, 14:32
Post #56





Group: Members
Posts: 107
Joined: 15-December 03
Member No.: 10468



QUOTE (gottkaiser @ Dec 4 2009, 04:08) *
QUOTE (GeorgeFP @ Dec 4 2009, 12:27) *
To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

I would like to use fpFLAC2MP3. But I'm not able to build the project. I'm just not into this subject.
Maybe somebody smart who owns all the needed software likes to build the project and upload it here in the forum? rolleyes.gif

That would be very appreciated.


Any chance the built files could be hosted on Rarewares?
The "old" fpMP3Enc is already there.
http://www.rarewares.org/mp3-others.php
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 7 2009, 19:55
Post #57





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



A new version with two bug fixes is available for download:

64-Bit version
32-Bit version (not supported)

Bug#1: Application hung (became idle) sometimes
Bug#2: Memory access violation

Tomorrow I will start work on the -V option and the replay gain feature.
Go to the top of the page
+Quote Post
JacksonGrey
post Dec 8 2009, 00:59
Post #58





Group: Members
Posts: 122
Joined: 5-February 09
Member No.: 66629



I keep getting an error that a number of files 'failed to register'. Anyone else experiencing this? Files are installed into the Program Files directory, but any conversion fails in Foobar.

This post has been edited by JacksonGrey: Dec 8 2009, 01:00
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 8 2009, 07:14
Post #59





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (JacksonGrey @ Dec 8 2009, 00:59) *
I keep getting an error that a number of files 'failed to register'. Anyone else experiencing this? Files are installed into the Program Files directory, but any conversion fails in Foobar.

The Visual C++ runtimes are not included in the package. You can download them here:

32-Bit
64-Bit

You can also try to register the DLLs manually:
1. Open a command prompt as administrator
2. Go to the installation directory
3. Type
CODE
regsvr32 fpFLAC.dll
regsvr32 fpFLAC_SSE2.dll
regsvr32 fpFLAC_SSE4.dll
regsvr32 fpMP3.dll
regsvr32 fpMP3_SSE2.dll
regsvr32 fpMP3_SSE4.dll
regsvr32 fpStreamLib.dll
regsvr32 fpWave.dll
regsvr32 lFLAC.dll
regsvr32 lVorbis.dll

Go to the top of the page
+Quote Post
GeorgeFP
post Dec 8 2009, 13:12
Post #60





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



I've found the problem with the installer: Several files were missing in the packages so I had to rebuild them again. Sorry for that. The above links still work.

You may get an error when registering fpFLAC_SSE2.dll or fpFLAC_SSE4.dll. This happens when your system does not support SSE2 or SSE4. Just press "Continue" then.

The new installer has updated versions of fpFLAC and fpFLAC2FLAC where you can pass the -V (verify) option.
Go to the top of the page
+Quote Post
greynol
post Dec 8 2009, 17:34
Post #61





Group: Super Moderator
Posts: 10044
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I'd love to see this done with metaflac so that I can do quick replaygain scanning.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
NuSkooler
post Dec 23 2009, 18:16
Post #62





Group: Members
Posts: 24
Joined: 9-January 07
Member No.: 39448



I cannot wait for this to be ready for production so to speak!

Any chance @ GPU support?
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 24 2009, 12:47
Post #63





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (NuSkooler @ Dec 23 2009, 18:16) *
I cannot wait for this to be ready for production so to speak!

Well, the truth is, I don't know if fpFLAC has a future. The feedback is very, very poor.

QUOTE
Any chance @ GPU support?

I will add GPU support into my framework and if the fpFLAC project will be continued it will also support GPUs.
Go to the top of the page
+Quote Post
Bylie
post Dec 24 2009, 14:59
Post #64





Group: Members
Posts: 84
Joined: 14-July 02
From: Belgium
Member No.: 2593



QUOTE (GeorgeFP @ Dec 24 2009, 12:47) *
Well, the truth is, I don't know if fpFLAC has a future. The feedback is very, very poor.

Well to be honest I've been following all of the new projects with a lot of interest and am really impressed by the progress made. Who wouldn't be:
  • fpFLAC/MP3 makes much better use of multicore CPU's which are destined to be the future as CPU clockrates are stagnating for quite some time now. I just saw that also GPU support will be integrated, very nice!
  • FLACuda already leverages all the GPU processing potential currently being neglected in anything but games.
  • TAK which has made some nice improvements compressing audio even further.
  • Others which I probably forgot...
But, personally, I'm not very inclined to switch to a new, unproven piece of software even if it incorporates proven code and/or algorithms like FLAC, WavPack, LAME, ...

Maybe a better approach to this would be, to somehow try, to integrate these new projects into the existing, proven, codebase supervised and/or publicly supported by the original developers trying to maintain the same stability and quality of the original software. This might even be a different branch at first but I really think the "umbrella effect" could have a positive effect on the acceptance of the new code/projects by the users which ultimately will result in more useage and feedback.

I know this is sort of an idealistic vision because in reality it might just not be that easy due to a number of realworld problems/conflicts (licensing, time, ...) but I really think it could give these new projects a lot more visibility and it might even give the original software a new momentum.

This post has been edited by Bylie: Dec 24 2009, 15:24
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Dec 24 2009, 15:20
Post #65





Group: Developer
Posts: 702
Joined: 2-October 08
From: Ottawa
Member No.: 59035



Agree. Josh, if you are interested, i'm ready to spend some time helping to incorporate CUDA code.


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
skamp
post Dec 24 2009, 16:30
Post #66





Group: Developer
Posts: 1450
Joined: 4-May 04
From: France
Member No.: 13875



QUOTE (Gregory S. Chudov @ Dec 24 2009, 15:20) *
Josh, if you are interested, i'm ready to spend some time helping to incorporate CUDA code.

GPU encoding in flac through libcuda under linux would be ideal (awesome).


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
_mē_
post Dec 25 2009, 00:56
Post #67





Group: Members
Posts: 231
Joined: 6-April 09
Member No.: 68706



QUOTE (GeorgeFP @ Dec 24 2009, 12:47) *
QUOTE (NuSkooler @ Dec 23 2009, 18:16) *
I cannot wait for this to be ready for production so to speak!

Well, the truth is, I don't know if fpFLAC has a future. The feedback is very, very poor.

Well, there's some feedback I can give you. I installed it and deleted shortly after. I'm not an user, but I can tell you why.

1. For me FLAC is dead, it's too weak, so you came too late. Multithreaded wavpack or mac would probably make more sense.

2. The last command line that I used was
flac -8 -A tukey(0.5) -A flattop -r 0,8 -V
and I considered adding -p, but it was a bit too slow for the gain. Overall it's slightly stronger than -8.

If fpFLAC allowed me to compress slightly better with reasonable speed (and I wouldn't be shifting to TAK), I would use it, though probably with verification done by regular FLAC during the testing period.
However, fpFLAC doesn't accept these switches, so (I guess) it's even weaker than regular FLAC. For me - pointless.

3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.

4. No Uninstaller. That's extremely annoying. I guessed that I should use the installer, but if I didn't keep it (for these several minutes wink.gif ), I would just delete the program and have rubbish left in the registry and possibly other places too. And I wouldn't assume that everybody will guess, even considering that the target audience is rather advanced.

5. FLAC is (pendrive) portable, fpFLAC - I don't know. For me, it's very important. The only non-portable program that I found worth using is Visual Studio.
Go to the top of the page
+Quote Post
Qest
post Dec 25 2009, 01:02
Post #68





Group: Members
Posts: 109
Joined: 1-February 04
Member No.: 11678



It would be outstanding if the defacto flac tools had support for multiple cores (like fpFLAC) and gpu's (like FLACuda)! That would be the best of all worlds.
Go to the top of the page
+Quote Post
NuSkooler
post Dec 25 2009, 01:03
Post #69





Group: Members
Posts: 24
Joined: 9-January 07
Member No.: 39448



QUOTE (_mē_ @ Dec 24 2009, 16:56) *
1. For me FLAC is dead, it's too weak, so you came too late. Multithreaded wavpack or mac would probably make more sense.


I doubt this is any reason for this particular project to receive minimal attention from outsiders. FLAC is the defacto standard for lossless audio. Sure, WavPack & others have gained some ground (especially around here) but in the "real world" you're going to find FLAC. I think some of the reason is people simply do not know of this. I use FLAC on a every day basis and just found out about this a few days ago. Also, until it can replace flac.exe with confidence, obviously it won't be used by many.

QUOTE
3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.


I agree. Especially for "utility" type stuff. My suggestion: Statically link everything needed into a fpFLAC.exe. I believe this will make it much more open to the masses.
Go to the top of the page
+Quote Post
Zarggg
post Dec 25 2009, 05:02
Post #70





Group: Members
Posts: 560
Joined: 18-January 04
From: bethlehem.pa.us
Member No.: 11318



Will this program yield any benefit to a (single core) Pentium 4 with Hyperthreading enabled? My immediate thought would be "no", but I still want to ask.

I apologize if this seems like a dumb question, but it's not very obvious to me and I'm only just starting to learn about how multi-core processors work.
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 25 2009, 13:33
Post #71





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (Bylie @ Dec 24 2009, 14:59) *
But, personally, I'm not very inclined to switch to a new, unproven piece of software even if it incorporates proven code and/or algorithms like FLAC, WavPack, LAME, ...

Maybe a better approach to this would be, to somehow try, to integrate these new projects into the existing, proven, codebase supervised and/or publicly supported by the original developers trying to maintain the same stability and quality of the original software. This might even be a different branch at first but I really think the "umbrella effect" could have a positive effect on the acceptance of the new code/projects by the users which ultimately will result in more useage and feedback.

Believe me, if I had the chance to integrate my framework into the existing C code I would have done so. But multi-core programming is very different to serial programming and thus, I had to write fpFLAC from scratch.

Of course, you can wait until the original developers will add multi-core support to their projects. I don't know of any activities going in this direction, neither for FLAC nor for LAME.
Go to the top of the page
+Quote Post
Bylie
post Dec 25 2009, 18:43
Post #72





Group: Members
Posts: 84
Joined: 14-July 02
From: Belgium
Member No.: 2593



QUOTE (GeorgeFP @ Dec 25 2009, 13:33) *
Believe me, if I had the chance to integrate my framework into the existing C code I would have done so. But multi-core programming is very different to serial programming and thus, I had to write fpFLAC from scratch.

Of course, you can wait until the original developers will add multi-core support to their projects. I don't know of any activities going in this direction, neither for FLAC nor for LAME.

I really try to not underestimate the amount of work and research that goes into these projects and I can surely understand that saying something like "integration will yield benefit x and y" might sound much easier than actually doing it!
However, the problem will remain for projects like fpFLAC and others that they are seen more like a proof of concept and hence are more interesting for developers and testers which won't make a large userbase. This might actually not be a problem if the goal is to provide a proof of concept, but if the idea is to get more and more realworld usage/feedback I think somewhere along the line the proof of concept stage has to evolve into something more users will (want to) use. This is where the "umbrella effect" or "public developer support" might play a role as it gives visibility to these projects beyond this forum and also, kind of, assures the large group of endusers that it's OK to also use this for anything else than testing.

This post has been edited by Bylie: Dec 25 2009, 18:46
Go to the top of the page
+Quote Post
Fandango
post Dec 25 2009, 19:29
Post #73





Group: Members
Posts: 1549
Joined: 13-August 03
Member No.: 8353



Uhm, I would happily replace all single-threaded command line encoders right away, with multi-core + GPU utilizing encoders. It annoys me everytime I transcode a single large file, that more than half of my CPU is idle, while I am wating... waiting... waiting...
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 25 2009, 20:51
Post #74





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (NuSkooler @ Dec 25 2009, 01:03) *
QUOTE
3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.


I agree. Especially for "utility" type stuff. My suggestion: Statically link everything needed into a fpFLAC.exe. I believe this will make it much more open to the masses.

My "streaming framework" which is similar to DirectShow uses the registry to find the components/filters. I think this is a better approach than putting everything into a single file, especially when you have optimizations for different processor features (SSE, CUDA, Stream, ...)
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 25 2009, 22:11
Post #75





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (Bylie @ Dec 25 2009, 18:43) *
I really try to not underestimate the amount of work and research that goes into these projects and I can surely understand that saying something like "integration will yield benefit x and y" might sound much easier than actually doing it!
However, the problem will remain for projects like fpFLAC and others that they are seen more like a proof of concept and hence are more interesting for developers and testers which won't make a large userbase. This might actually not be a problem if the goal is to provide a proof of concept, but if the idea is to get more and more realworld usage/feedback I think somewhere along the line the proof of concept stage has to evolve into something more users will (want to) use. This is where the "umbrella effect" or "public developer support" might play a role as it gives visibility to these projects beyond this forum and also, kind of, assures the large group of endusers that it's OK to also use this for anything else than testing.

I agree with you. For me, it's ok if fpFLAC is seen as a proof of concept and the project can end here right away.
Go to the top of the page
+Quote Post

5 Pages V  < 1 2 3 4 5 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th October 2014 - 15:43