Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: lossyWAV 1.3.0 Delphi to C++ Translation Thread (Read 158245 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Tycho started the Delphi to C++ translation of lossyWAV in August 2012. After some time getting to grips with C++ and a bit of debugging, I started to develop lossyWAV 1.4.0.

Changelog:

lossyWAV v1.4.0 02/10/2014
  • Released
[!--sizeo:1--][span style=\"font-size:8pt;line-height:100%\"][!--/sizeo--]lossyWAV beta 1.3.1m 04/08/2014
  • Drop-dead-date now end of September 2014.
lossyWAV beta 1.3.1k 02/06/2014
  • Drop-dead-date now end of July 2014.
lossyWAV beta 1.3.1j2 01/04/2014
  • Drop-dead-date now end of May 2014.
  • Crash fixed.
lossyWAV beta 1.3.1j 31/03/2014
  • Drop-dead-date now end of May 2014.
  • Code tidy-up.
lossyWAV beta 1.3.1h 31/01/2014
  • Drop-dead-date now end of March 2014.
  • Code tidy-up.
lossyWAV beta 1.3.1g 01/12/2013
  • Drop-dead-date now end of January 2014.
  • Code tidy-up.
  • Modifications to --shaping altfilter.
lossyWAV beta 1.3.1f 15/10/2013
  • Drop-dead-date still end of November.
  • Bug hunt attempt #1 for non-specific crash.
lossyWAV beta 1.3.1e 07/10/2013
  • Drop-dead-date now end of November.
  • Additions to --feedback parameter: "--feedback [n]" and "V, verbose". The addition of the numeric parameter (0.0 <= n <= 10.0) allows the user to increase the feedback sensitivity. Verbose increases the level of detail in the feedback output.
  • A shorter FFT analysis has been introduced: 16 samples. This is now the first additional FFT analysis to be added (using "-a 4" or "--analyses 4"). Other FFT lengths are added in increasing numerical order.
  • Bug fixes in output and some variable structural changes.
lossyWAV beta 1.3.1d 15/09/2013
  • Drop-dead-date now end of October.
  • Additions to --feedback parameter: "r, round <n>" reduces the permitted deviation between expected added noise due to rounding and actual; "n, noise <n>" reduces the added noise level due to adaptive noise shaping; "a, aclips <n>" defines the permitted number of exceedences of the permitted adaptive noise shaping limiting level; "A, alevel <n> reduces the permitted adaptive noise shaping limiting level;
  • Slight change to rounding clipping behaviour. Only clips introduced after scaling will be used when determining whether number of bits to remove should be reduced due to clipping.
lossyWAV beta 1.3.1c 04/08/2013
  • Drop-dead-date now end of September. Apologies for any inconvenience caused by the introduction of an expiry date for beta versions.
lossyWAV beta 1.3.1b 20/06/2013
  • Error in correction file re-combination (was restricted to 24-bit files) corrected.
lossyWAV beta 1.3.1a 18/06/2013
  • Improvements to WAVE64 and RF64 compatibility;
  • Correction files work again for WAV and also WAVE64 and RF64;
  • Dependency on pthreadgc2.dll removed.
lossyWAV beta 1.3.0o 11/06/2013
  • Introduction of WAVE64 and RF64 compatibility. lossyWAV will now read and write both formats in addition to WAV.
lossyWAV beta 1.3.0n 30/03/2013
  • experimental parameter --feedback introduced to modify bit removal process to include an added noise check. Seems to work well in conjunction with "--maxclips 0" or "--maxclips 1" at lower quality settings.
lossyWAV beta 1.3.0m 25/03/2013
  • Removed experimental new parameter --noisebtr.
  • --interp-curve reinstated.
  • experimental parameter --altfilter introduced to modify adaptive noise shaping method.
lossyWAV beta 1.3.0k3 08/03/2013
  • Introduction of experimental new parameter --noisebtr which applies a new method of detemining bits to remove. Well, I say new - SebastianG did point me in this direction *quite* some time ago. Basically, the creation of the noise shaping filter for each channel outputs a numerical value that I have finally used to determine the number of bits to remove from that codec-block channel. It even works on (non-clipping) tonal samples. It should be noted that this is a complete departure from the original method - no searching for the lowest signal in the FFT output bins. Limited tuning so far - however the presets should give approximately the same resultant FLAC output bitrate from --portable up to --insane. Below --portable, I have made the quality settings more aggressive and --extraportable results in a FLAC output bitrate of approximately 283kbit/s. This is quite a drop from the 308 to 311kbit/s previously. If anyone is prepared to spend the time on some testing, that would be very much appreciated. I think that --extraportable is a bit *too* aggressive at the moment, but that is more of a feeling than anything else.
  • --interp-curve removed at this time.
  • beta 1.3.0k2 issued due to omission of --noisebtr from text output parameter list (used in FACT chunk and log file).
  • beta 1.3.0k3 issued due to last minute fix for bits-per-sample not equal to 16.
lossyWAV beta 1.3.0j 08/02/2013
  • Further experimental modification to the adaptive noise shaping routine, backed off a little but now works well with >96kHz samplerate
lossyWAV beta 1.3.0i 30/01/2013
  • Further experimental modification to the adaptive noise shaping routine. NB: Testing has shown that this does not work well with >96kHz samplerate!!.
  • Bug-fix: now processes >16-bit properly - unexpected integer overflow discovered where floating point was thought to be used.
lossyWAV beta 1.3.0h 23/01/2013
  • Further experimental modification to the adaptive noise shaping routine. FLAC encode (-5 -b 512) from "-q X -i" results in 302kbit/s (compared to 309 kbit/s for vanilla --extraportable).
lossyWAV beta 1.3.0g 18/01/2013
  • Experimental modification to the adaptive noise shaping routine, new parameter -i, --interp-curve; adds optional cubic curve interpolation in lieu of linear interpolation when determining target spectral shape for each noise shaping filter. When encoded in FLAC (-5 -b 512) using the --extraportable preset, my 10 album test set results in 304kbit/s (compared to 309 kbit/s for vanilla --extraportable).
lossyWAV beta 1.3.0f 26/10/2012
  • First attempt at FFMPEG compatibility;
lossyWAV beta 1.3.0e 24/09/2012
  • More rounding issues found and amended;
  • Output is hopefully bit identical with reference output (2nd attempt....);
lossyWAV beta 1.3.0d 18/09/2012
  • Bit removal chain error identified: C++ inherently rounds 0.5 differently from Delphi, Delphi uses Banker's rounding (round to even);
  • Banker's rounding implemented;
  • Output is now bit identical with reference output (caveat: from testing so far). [edit] Found not to be at --portable. Bug hunting continues.... [/edit]
lossyWAV beta 1.3.0c 15/09/2012
  • Piped I/O implemented - please report problems;
  • Most bugs in bit-removal chain removed - output not yet bit identical;
  • Internal FFT routines optimised.
[/size]
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)


lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #2
Is the port done publicly, like on github or similar? This community has some very knowledgeable C++ hackers, so it might speed the process and help with finding and fixing bugs.
It's only audiophile if it's inconvenient.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #3
JAZ, Thanks, I was aware of the C# translation of version 1.0, but had missed out your java translation. Nice. There are a lot of changes in 1.3 though, and much of the work is to convert the I/O handling and runtime linking with fftw3-3 library. First step is something that compiles with gnu g++ on windows. Next would be VS2010, and then Linux/Mac g++, but that could easily someone else do.

Kohlrabi, I haven't thought about using github yet.  My initial thought was just to get most of the code to compile, and then upload the code here. Useful help would be that people browsed through it and verified that the .pas and the .h/.cpp files does the same thing, but I wouldn't mind if people worked on it too.

I think that modern optimized C++ compilers (both 32 and 64 bit) are able to improve on speed a fair bit, but portability and accessibility is as important.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #4
Hi Tycho,

How are you getting on? Impatient Inquiring minds would like to know....
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #5
It's progressing quite well.  Many of the files are files are basically compiling now, but there are still some unresolved functions, e.g. the Delphi Write() function must be emulated, or all the calls to it must be rewritten. Also some of the IO routines need some work, as I mentioned.

I can post the code as it is tomorrow evening, so you can have a feel of it. Later I will come back with few questions for you, and tell a few things I've done during translation.

E.g. You have a few small functions in assembly only, like the ArcTan_Complex().  Shouldn't that return a complex number and not a float? On googling it, I found an implementation: (C++11 has it in the standard lib now)

Complex atan(Complex c)
{
  Complex log_v = log( Complex(1.0 - c.imag(), c.real()) / Complex(1.0 + c.imag(), -c.real()) );
  return Complex( log_v.imag() * 0.5, -log_v.real() * 0.5 );
}

The float you are returning is maybe the magnitude of the complex number returned here?

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #6
Code: [Select]
function ArcTan_Complex(const X: tDComplex): Extended;
asm
                FLD       X.im
                FLD       X.re
                FPATAN
End;

According to google it is arctg(x.im/x.re) so basically it is C function atan2(x.im, x.re).
...IMHO.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #7
Apologies - you've found some of the "dirtier" code in lossyWAV....

The ArcTan_Complex function is, as lvqcl has rightly said, equivalent to a two input atan function.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #8
Once we have C++ version can we expect nice speed up after compiling with Intel Compiler with enabled auto vectorization and auto paralyzation? I must admit that delphi's version is rather slooow. It would be awesome if code was multithreaded and SIMD friendly.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #9
I haven't studied enough how the data is processed, but I believe that the code must be made more reentrant, so that central functions relies exclusively on own stack data, and not global data structures (except readonly) as today. This opens the door for parallel computation if the algorithm allows for it, which I believe it does.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #10
A simple parallelism would be to process each channel concurrently as channels are independent (even if linked then that linking of bits_to_remove would occur after all the FFT analyses have been completed for a given codec block).
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #11
A simple parallelism would be to process each channel concurrently as channels are independent (even if linked then that linking of bits_to_remove would occur after all the FFT analyses have been completed for a given codec block).

that would be a good start. 2x faster proccesing on stereo track and even more for typical 5.1 movie sound track (assuming that user has atleast quad core).

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #12
Another would be to have the FFT analyses run in parallel - each 512 sample codec block (44.1kHz / 48kHz samplerate) has 1x 1024 sample, 16x 64 sample and 32x 32 sample FFT analyses carried out on it (per channel).

[edit] can't count, apparently.... [/edit]
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #13
I have noticed that original LossyWav 1.3 does not support .wavs larger than 4GB. My wav file is 5.2GB  and lossyWAV sees only 1.2 GB. LossyWav also does not support wave64 format either.


Due to the same reason pipes do not work either with large flacs

Code: [Select]
flac.exe -d "E:\Avatar.2009.BluRay.REMUX.1080p.AVC.DTS-HD.MA5.1.flac" --stdout --silent | lossywav.exe - --stdout | flac.exe - -b 512 -8 --silent -o "E\lossy.flac"


i think that new version should have something like --readtoeof or --ignorelength switch like other encoders (Aften AC3 , FhGAACEnc or OpusEnc)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #14
A specification compliant WAV file can be no larger than 4GB (minus a few bytes for header and format information).

Maybe looking at RIFF64 or BWF would be a better way forward to cope with very large files.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #15
Without --ignorelength switch lossyWav is useless for movie sound tracks. I can't even use pipe for direct processing.

BTW. Do you know maybe why your sources won't compile in delphi 7 pro?

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #16
You could split the soundtrack into chapters, process with lossyWAV and then recombine using foobar2000.

I don't know about Delphi 7 Pro - I have been using Turbo Delphi Explorer 2006.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #17
Tycho,

Are there any specific compiler options that I should be using for compilation of the project?

I have downloaded the Dev-CPP IDE with MinGW 4.6.0.2 - primarily because I wanted a portable mode. Quite liking it and have started "playing" with it to get my head round C++.

On the Write() issue, would fprintf() be better? It would certainly make formatting the output much easier. I have got v0.3 to compile as far as nOutput.cpp.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #18
Nick,
Good to hear you're progressing with c++. Yes, the old ANSI C printf (stdio.h) is in many ways easier and more compact to use, although less type safe (it normally does not check arguments in the printf statements with the formatting string). Personally I would prefer to use the current standard which is the fstream/iostream interface.

To be blunt, DEV Cpp has been dead for years, and it's Windows only (written i Delphi actually).  There are better alternatives as I have mentioned earlier. For cross platform I would first recommend Qt Creator (available on Win,Linux,OS X). It is meant for Qt apps, but you may easily develop pure C++ projects without Qt and qmake. It supports many compilers, .e.g Visual Studio (Express) and GNU Mingw. You may download Qt libraries 4.8.2 (not the whole Qt SDK) for the qmake and cross platform GUI ++ libs. As a second alternative you can look at CodeLite, which is a one-man project only, then Code::Blocks which is similar.

I am going to configure Qt Creator myself with Mingw to fix the compilation issues there. It behaves different than VC++. I have currently Mingw 4.7.0 on my machine so there may be differences from 4.6.2. Don't worry about the Write() now, I will take care of that one when we get there.

This is probably information overload, so take it easy.

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #19
Updated executable later tonight - huge thanks to Tycho for getting me off my butt to delve into C++ with his translation of the Delphi source code.

A few issues found and corrected, not least the in FFT code - there were a few issues in there (int equates to FLOOR; >> is shift-arithmetic-right, not a simple shift right; log is ln, not log10....).
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #20
what is still missing in current c version vs original?

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #21
A bit of accuracy - there's a bug somewhere in the chain after the FFT analyses that still has to be tracked. Adaptive noise shaping is confirmed to be working though.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #22
lossyWAV beta 1.3.0b attached. [edit] Superseded. [/edit]

Changelog:

Internal FFT routines now work.

To do list:

Piped input / output.
Track down accuracy error post FFT analyses.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #23
lossyWAV beta 1.3.0b attached.

Changelog:

Internal FFT routines now work.

To do list:

Piped input / output.
Track down accuracy error post FFT analyses.

after adding pipes could you also add --ignorelength switch?

lossyWAV 1.3.0 Delphi to C++ Translation Thread

Reply #24
Is there any reason for custom FFT? Why not something like KISSFFT or something?