IPB

Welcome Guest ( Log In | Register )

8 Pages V  < 1 2 3 4 > »   
Reply to this topicStart new topic
lossyWAV 1.3.0 Delphi to C++ Translation Thread, Added noise linear PCM bitdepth reduction method.
mezenga
post Sep 8 2012, 04:32
Post #26





Group: Members
Posts: 121
Joined: 10-May 03
Member No.: 6509



QUOTE (mudlord @ Sep 7 2012, 16:37) *
Is there any reason for custom FFT? Why not something like KISSFFT or something?

At Delphi“s development time the reason seemed to be the lack of options.
But this C++ port surely is a good opportunity to review if it remains the better choice.

And here is some small lossyWAV FFT implementation history:
lossyWAV alpha v0.3.12 - Special thanks: Dr. Jean Debord for the use of TPMAT036 uFFT & uTypes units for FFT analysis

lossyWAV v0.6.4 RC1 - Special thanks: Don Cross for the original Pascal source for the FFT algorithm used.

lossyWAV 1.1.0b - Special thanks: Don Cross for the Complex-FFT algorithm used[/url]

Change log 1.1.4b: 14/05/09: FFT (Delphi and assembler) further optimised. Radix-4 FFT implemented in assembler and Delphi and Radix-8 in Delphi. Significant speedup of Delphi FFT throughput.

This post has been edited by mezenga: Sep 8 2012, 04:45
Go to the top of the page
+Quote Post
Nick.C
post Sep 8 2012, 13:20
Post #27


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



I'll have a look at KissFFT - it may be faster.

I also need to try to get to grips with multi-threading to see if there is any way to speed up the processing.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
lvqcl
post Sep 8 2012, 13:23
Post #28





Group: Developer
Posts: 3469
Joined: 2-December 07
Member No.: 49183



There's also PFFFT library: https://bitbucket.org/jpommier/pffft But it is for single-precision floats.

This post has been edited by lvqcl: Sep 8 2012, 13:25
Go to the top of the page
+Quote Post
Nick.C
post Sep 8 2012, 13:28
Post #29


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



I know that single precision would be faster but after trying it out earlier in the development of 1.3.0, I would much prefer to stick to double-precision.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
lvqcl
post Sep 8 2012, 13:55
Post #30





Group: Developer
Posts: 3469
Joined: 2-December 07
Member No.: 49183



SoX uses fft4g.c from Ooura 1D FFT
Go to the top of the page
+Quote Post
Destroid
post Sep 9 2012, 10:50
Post #31





Group: Members
Posts: 555
Joined: 4-June 02
Member No.: 2220



Being unaware of KissFFT, I did a brief overview and noticed some factors:

- FFTW has benchmarks showing (on x64) having notably higher performance than KissFFT (the former program being version 3.3.x, the latter program's version I did not observe);

- That KissFFT was not present on all of those benchmarks, possibly having to do with limitations on certain kinds of double-precision inputs;

- That KissFFT is purported to have simpler implementation and/or structure.

I am not an expert in areas of DSP and FFT and wanted to report my impressions and conclusions. I enjoy watching the development process as I also hope to gain knowledge. I congratulate and admire the developers' efforts (and HA, of course).


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
mezenga
post Sep 9 2012, 15:59
Post #32





Group: Members
Posts: 121
Joined: 10-May 03
Member No.: 6509



QUOTE (Destroid @ Sep 9 2012, 01:50) *
- FFTW has benchmarks showing (on x64) having notably higher performance than KissFFT (the former program being version 3.3.x, the latter program's version I did not observe);

FFTW is optionally supported since lossyWAV 1.1.4c (requires libfftw3-3.dll and --fftw switch) but the main discussion here is about the internal FFT function.
Until now FFTW saved 10-20% processing time: enough to became an option but not enough adding 500Kb in the download package.
Lets wait Nick and Tycho get some benchmarks from this C++ port.
Go to the top of the page
+Quote Post
Nick.C
post Sep 9 2012, 19:06
Post #33


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



The C++ executable is *much* bigger than the Delphi one (1716kiB vs. 168kiB).

I am in the process of implementing a radix-4 IFFT while optimising the other FFT routines for speed. I think that there is some speed to be had. Also, the minGW compiler allows optimisation for various different processors and instruction sets. These do make an appreciable difference.

Main target is to get piped I/O working and to track down the accuracy bug previously mentioned.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
tycho
post Sep 9 2012, 22:24
Post #34





Group: Members
Posts: 345
Joined: 5-August 03
Member No.: 8183



Even though much of the point of translating to C++ was the possible optimization benefits, I think correctness is still the most important at this point. I believe there are still issues I have mentioned elsewhere that needs to be addressed, or have they been handled? Also, in getParams() an if-statement is missing:
CODE
size_t found = parameters.wavName.find_last_of("/\\");
if (found != std::string::npos)
{
    parameters.WavInpDir = parameters.wavName.substr(0, found + 1);
    parameters.wavName = parameters.wavName.substr(found + 1);
}

For piped IO, make sure to set input/output stream to binary mode on windows.
CODE
#ifdef _WIN32
#include <cstdio>
#include <io.h>
#include <fcntl.h>
#endif
...
if (parameter.STDINPUT || parameter.STDOUTPUT) {
    #ifdef _WIN32
    _setmode(_fileno(stdin), _O_BINARY);
    _setmode(_fileno(stdout), _O_BINARY);
    #endif
    ...
}

This will also make std::cin and std::cout binary, i.e. does not translate linefeed char (LF) to CRLF - leaves it as LF. Works both under g++ and VS.

As for executable size, you may try -Os compiler flag and then "strip lossyWAV.exe". With Visual Studio, the executable doesn't get very large.
Go to the top of the page
+Quote Post
Nick.C
post Sep 10 2012, 21:38
Post #35


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Thanks for those Tycho.

I've started on piped I/O - it's taking a bit of reconfiguration but will get there in the end.

I will also go back through to check for other previously mentioned issues.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Nick.C
post Sep 12 2012, 18:39
Post #36


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Piped I/O is now in testing phase - I processed my 10 album test set in foobar2000 without incident.

I've tracked down two problems in the bit-removal chain but am still getting slightly different results than the "reference" Delphi version. More bug-hunting to do there then.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Atak_Snajpera
post Sep 13 2012, 18:37
Post #37





Group: Members
Posts: 28
Joined: 16-August 12
Member No.: 102388



have you decided yet what multithreading method will you choose for extra speedup?

do you even consider reading wavs to the end file? ( 4gb+ wavs )

This post has been edited by Atak_Snajpera: Sep 13 2012, 18:40
Go to the top of the page
+Quote Post
Nick.C
post Sep 13 2012, 19:19
Post #38


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



I have not started on multi-threading yet. I will go through the debugging process until I am content that the output is a close to the output of the reference version as it can be.

I am considering adding "--ignore-chunk-sizes" functionality, but I don't yet know how best to go about it. It will have to wait until I have finished the debugging process.

[edit] English language fail.... [/edit]

This post has been edited by Nick.C: Sep 13 2012, 19:20


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Atak_Snajpera
post Sep 13 2012, 20:03
Post #39





Group: Members
Posts: 28
Joined: 16-August 12
Member No.: 102388



i keep asking about ignore chunk size switch because i plan to add lossywav preprocessing in ripbot264.

This post has been edited by Atak_Snajpera: Sep 13 2012, 20:09
Go to the top of the page
+Quote Post
Nick.C
post Sep 13 2012, 21:07
Post #40


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Ah! Now it makes sense to me. In that case, I will confirm that I will attempt to develop the --ignore-chunk-sizes functionality (when I've finished initial debugging wink.gif).


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Atak_Snajpera
post Sep 13 2012, 21:30
Post #41





Group: Members
Posts: 28
Joined: 16-August 12
Member No.: 102388



great and thank you!
Go to the top of the page
+Quote Post
Nick.C
post Sep 15 2012, 12:00
Post #42


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



lossyWAV beta 1.3.0c attached to post #1 in this thread. [edit] Superseded. [/edit]

This post has been edited by Nick.C: Sep 18 2012, 21:56


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Atak_Snajpera
post Sep 15 2012, 18:36
Post #43





Group: Members
Posts: 28
Joined: 16-August 12
Member No.: 102388



Speed comparison
Q6600@3Ghz
All read/write done on RAMDISK 2GB

CODE
flac.exe -d "G:\original.flac" --stdout --silent | G:\lossywav.exe - --stdout | G:\flac.exe - -b 512 -8 --silent -o "G:\lossy.flac"


DELPHI version - 11.48x
C++ version - 14.88x
Go to the top of the page
+Quote Post
Nick.C
post Sep 15 2012, 18:44
Post #44


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



That's certainly encouraging smile.gif!


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Atak_Snajpera
post Sep 15 2012, 18:46
Post #45





Group: Members
Posts: 28
Joined: 16-August 12
Member No.: 102388



Ability to process channels independently each in separate thread should give also nice extra speedup. Especially for 6ch movie soundtracks.

I wonder if compiling with Intel Compiler we could get even higher speed than your build.

This post has been edited by Atak_Snajpera: Sep 15 2012, 18:48
Go to the top of the page
+Quote Post
[JAZ]
post Sep 15 2012, 19:28
Post #46





Group: Members
Posts: 1798
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



I can confirm that higher speed will be possible:

(Core 2 Duo at 1.5GHz)

Delphi : 5.75x
GCC : 7.42x
VC x86 broken : 6.73916x
VC x64 broken : 7.14685x

Delphi FFTW : 7.49x
GCC FFTW : 8.22x
VC x64 FFTW broken : 8.86619x

VC broken means a modified compile of the sources released as v05 by tycho that I've compiled with Visual Studio 2008. x86 compile is with settings: full optimizations and SSE2. (With FFTW, the output is a bit more like GCC, but is still broken.).
Oh, and the size is, between 350KB and 400KB without requiring the msvc runtimes (I've linked it statically. Else it would be around 200KB and require the VS 2008 runtimes).

@Nick: are you waiting to have a completely working version before releasing the C++ sources? I'm asking so that I don't fix things that you've already fixed.
Also, I've seen that the GCC version shows correctly the --bitdist, but --blockdist is completely different. Not sure if that shows a problem in the displaying/storage of those values, or if it is the algorithm. I guess most of the switches you added about statistics should help on debugging this port smile.gif
Go to the top of the page
+Quote Post
Nick.C
post Sep 18 2012, 18:56
Post #47


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



lossyWAV beta 1.3.0d attached to post #1 in this thread. [edit] Superseded. [/edit]

This post has been edited by Nick.C: Sep 24 2012, 20:27


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
mezenga
post Sep 19 2012, 17:32
Post #48





Group: Members
Posts: 121
Joined: 10-May 03
Member No.: 6509



QUOTE (Nick.C @ Sep 18 2012, 09:56) *
lossyWAV beta 1.3.0d attached to post #1 in this thread.

Tested against lossyWAV 1.3.0 delphi version for bit identical output with internal FFT and FFTW.

- standard quality setting: OK (bit identical).

- portable quality setting: KO (not bit identical).
Go to the top of the page
+Quote Post
Nick.C
post Sep 19 2012, 17:39
Post #49


lossyWAV Developer


Group: Developer
Posts: 1815
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



.... ok. Thanks for the testing - I'll go bug hunting again.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
punkrockdude
post Sep 19 2012, 23:16
Post #50





Group: Members
Posts: 268
Joined: 21-February 05
Member No.: 20022



I really look forward to try this out on Linux one day. I really hope Linux drivers (especially proffessional audio interfaces) will soon be a reality so that I can use only Linux. Since installing an i3 i5 i7 patched kernel I am so impressed with Linux responsiveness. Why do I talk about Linux? Sorry guys smile.gif
Go to the top of the page
+Quote Post

8 Pages V  < 1 2 3 4 > » 
Reply to this topicStart new topic
2 User(s) are reading this topic (2 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 29th December 2014 - 07:26