IPB

Welcome Guest ( Log In | Register )

27 Pages V  « < 2 3 4 5 6 > »   
Closed TopicStart new topic
lossyWAV 1.2.0 Development Thread, Added noise WAV bitdepth reduction method
Gregory S. Chudo...
post Nov 18 2008, 21:20
Post #76





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



Thanks.

QUOTE (Nick.C @ Nov 18 2008, 23:05) *
Christopher's decoder outputs 24-bit WAV files (for decoded HDCD output).

In fact, it outputs 24-bit WAV files with 4 lower bits set to zero.

I still don't understand, can (or should) i somehow instruct lossyWAV that 4 lower bits are zero, and will it make any difference? In other words, does lossyWAV support wierd bitsPerSample values, or only fixed set of 16/24?

This post has been edited by Gregory S. Chudov: Nov 18 2008, 21:54


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
pdq
post Nov 18 2008, 22:09
Post #77





Group: Members
Posts: 3407
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



Any low bits that are already zero will remain zero. What lossy wav does is, when appropriate, zero out additional lower bits to make lossless compression more effective.
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Nov 18 2008, 22:39
Post #78





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



QUOTE (pdq @ Nov 19 2008, 00:09) *
Any low bits that are already zero will remain zero. What lossy wav does is, when appropriate, zero out additional lower bits to make lossless compression more effective.

Thanks a lot, i really appreciate the answers but those are the things that i already know. What i wanted to know, is does lossyWAV support 20-bit input natively.

Let me explain:
If i take 16 bit wav, and turn in into 24 bit wav by adding 8 zero bits, then process with lossyWAV, then turn it in 16 bit wav again by removing 8 lower bits, i'm not sure i will get the same result as if i processed the 16 bit input directly.

16 bit WAV -> add 8 bits -> lossyWAV -> remove 8 bits == 16 bit WAV -> lossyWAV?

If yes, than it makes no difference if the input is 20 bit or 24 bit with 4 wasted bits.
But if no, than it makes a difference if 20 bit input is supported natively.

This post has been edited by Gregory S. Chudov: Nov 18 2008, 22:44


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
Nick.C
post Nov 19 2008, 08:17
Post #79


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



A 20-bit sample in a 24-bit container will already be "left justified" as required by the WAV specification, i.e. effectively multiplied by 16, therefore lowest 4 bits are going to be zero.

16>24>lossyWAV>16 will probably not equal 16>lossyWAV, additionally, if the lossyWAV>16 step includes dither then the carefully zeroed lsb's will disappear and the wasted-bits compression advantage will be lost.

[edit] lossyWAV supports 9-bit to 32-bit integer samples in 16, 24 and 32-bit signed integer "containers", exactly the same as the WAV specification. [/edit]

This post has been edited by Nick.C: Nov 19 2008, 08:20


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Nov 20 2008, 15:42
Post #80





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



QUOTE (Nick.C @ Nov 19 2008, 10:17) *
A 20-bit sample in a 24-bit container will already be "left justified" as required by the WAV specification, i.e. effectively multiplied by 16, therefore lowest 4 bits are going to be zero.

16>24>lossyWAV>16 will probably not equal 16>lossyWAV, additionally, if the lossyWAV>16 step includes dither then the carefully zeroed lsb's will disappear and the wasted-bits compression advantage will be lost.

[edit] lossyWAV supports 9-bit to 32-bit integer samples in 16, 24 and 32-bit signed integer "containers", exactly the same as the WAV specification. [/edit]


Thanks.
Ok, i ported lossyWAV to C#, and will try to fool around with 20-bit input.
I can also send my code when i tidy it up a little, if you are interested.


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
Nick.C
post Nov 20 2008, 16:06
Post #81


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Yes, thanks.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Nov 20 2008, 23:14
Post #82





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



Here it is:

http://www.hydrogenaudio.org/forums/index....showtopic=67418

Tested to be bit identical with 1.1.1e on at least one file, at least with default quality setting smile.gif


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
lvqcl
post Nov 21 2008, 01:18
Post #83





Group: Developer
Posts: 3383
Joined: 2-December 07
Member No.: 49183



Processing 1-hour wav (16/44):
LossyWav.exe: 73 sec;
LossyWavSlow.exe: 239 sec;
LossyWavSharp.exe: 144 sec;

Cute. And yes, LossyWavSharp and LossyWavSlow produced identical output.
Go to the top of the page
+Quote Post
Nick.C
post Nov 21 2008, 08:47
Post #84


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Gregory S. Chudov @ Nov 20 2008, 23:14) *
Here it is:

http://www.hydrogenaudio.org/forums/index....showtopic=67418

Tested to be bit identical with 1.1.1e on at least one file, at least with default quality setting smile.gif
Gregory,

Many thanks for taking the time to port the core functionality of lossyWAV from Delphi to C#. The speed of this first version is impressive!

Looking at the code for the FFT functions - are you using a trick to only have to create one array of A(Re,Im) values by shifting the index by the bitlength difference of the FFT length in question and the maximum FFT bitlength?

Rógerio Brito (rbrito on these forums) is interested in a C port - how different is C# to C (I *really* don't know anything about C)?

Well done again!

Nick.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Nov 21 2008, 10:47
Post #85





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



QUOTE (Nick.C @ Nov 21 2008, 10:47) *
Looking at the code for the FFT functions - are you using a trick to only have to create one array of A(Re,Im) values by shifting the index by the bitlength difference of the FFT length in question and the maximum FFT bitlength?

Yep. I'm also using a similar trick for reversedbits table.

QUOTE (Nick.C @ Nov 21 2008, 10:47) *
Rógerio Brito (rbrito on these forums) is interested in a C port - how different is C# to C (I *really* don't know anything about C)?

It's still a lot of work, but about twice easier now - the basic operators are the same in C and C#, as are the basic concepts, such as arrays indexing. Most of the problems i had at first were caused by pascal-style arrays indexing (often starting at 1 instead of 0). With these obstacles removed, it's now only a matter of time to create a C port. Anyone who is familiar with C can read C# code. I hope someone will find a time to do this smile.gif

UPD: found a small bug in my WriteChunk function from AudioCodecsDotNet.cs, the chunks were not aligned on the word boundary. Corrected version is in CUETools SVN.

This post has been edited by Gregory S. Chudov: Nov 21 2008, 23:32


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Nov 22 2008, 16:29
Post #86





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



I added lossyWAV support to CUETools.

New CUETools also include command line lossyWAV utility.

Binaries can be found here, and source code is here.

LossyWAVDotNet is a C# class library which implements the algorithm,
AudioCodecsDotNet is a C# WAV reader/writer library
LossyWAVSharp is a command line utility which uses them.

Those 3 projects can be built independently of other projects in CUETools SVN.

Changes to the first version: class library interface improved, fft very slightly optimized and command line utility supports some basic command line options.


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
Nick.C
post Nov 22 2008, 19:12
Post #87


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



It looks like I should "hang up" Delphi and get my head around C....

Well done again!


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Neasden
post Nov 22 2008, 19:24
Post #88





Group: Banned
Posts: 185
Joined: 1-July 08
Member No.: 55148



Did people give up on ABX'ing LossyWAV? I didn't see usual ABX tests anymore. I hope it's that good!
Go to the top of the page
+Quote Post
Nick.C
post Nov 22 2008, 21:11
Post #89


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



--standard (-S or -q 5) appears, from all of the individual testers' responses appears to be transparent.

--portable (-P or -q 2.5) also appears to be transparent from the limited testing done on it so far.

My HTPC local copy of my full collection is --portable lossyFLAC, and comes in at about 372kbps, down from 880kbps for lossless FLAC. I have not yet heard an artefact in that collection - however I am not an ABX tester....


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
halb27
post Nov 22 2008, 23:39
Post #90





Group: Members
Posts: 2435
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Neasden @ Nov 22 2008, 20:24) *
Did people give up on ABX'ing LossyWAV? I didn't see usual ABX tests anymore. I hope it's that good!

New ears are always welcome!
I did a lot of ABX testing before, but now that lossyWAV has been in a mature state since a long time there is no sense to continue testing. To me and those samples I tested lossyWAV is perfect. But this doesn't mean that in the universe of music there may be problems showing up. That's why it would be nice if we had new testers.

In case you like to distribute I suggest you do it with the --portable quality. In theory at least there should be samples with --portable being not transparent. If such samples should show up it is important to check whether the --standard quality is fine with them or not.

This post has been edited by halb27: Nov 22 2008, 23:41


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
singaiya
post Nov 23 2008, 00:10
Post #91





Group: Members
Posts: 365
Joined: 21-November 02
Member No.: 3830



I gave up ABX testing it. It was really difficult on --portable, even though I came very close on a few samples in my transcoding test.
Go to the top of the page
+Quote Post
[JAZ]
post Nov 23 2008, 13:08
Post #92





Group: Members
Posts: 1783
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



QUOTE (Gregory S. Chudov @ Nov 22 2008, 16:29) *
LossyWAVDotNet is a C# class library which implements the algorithm,
AudioCodecsDotNet is a C# WAV reader/writer library
LossyWAVSharp is a command line utility which uses them.


I want to add that i've started a Java port from those C# sources.
The performance will be lower, for sure, and i'll have to recheck limits here and there ( for java, everything is signed, unsigned values do not exist. multi-level arrays are not matrixes, but arrays with references to other arrays. Casting can not be freely done like in C/C++/C#.)

I am trying to write it in the most correct way, as being more a reference than a best-of. (I've extracted the FFT parts into its own class, for example. I'm making constants in its own file, I'm removing redundant information and try to simplify the workflow).

I guess i will need still two or three days to complete the port. I'll post it here when done.
Go to the top of the page
+Quote Post
Nick.C
post Nov 23 2008, 21:09
Post #93


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Thanks for your effort [JAZ], it's appreciated.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
[JAZ]
post Nov 30 2008, 20:33
Post #94





Group: Members
Posts: 1783
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



Some updates on the port to Java:

I have a working code already, based mostly on the C# sources and to a lesser extent to the delphi sources or the web when something didn't look quite right/clear enough.

I don't have yet a .jar that could be tested, but i'll look into making one soon.

Currently:

It is a lot slower than the C# sources. ( by two orders of magnitude). This is because i've removed some of the lookup tables and implemented the FFT methods as they would be by the book.
Of course, i am in the process to add those tables back, in a way that could reflect what they really are and why they are there.

Another thing that i've discovered so far is that while the output seems to produce similar results, it does not produce the same ones than lossyWavSharp.
I've processed a .wav with both programs, and they play fine in foobar, but the one generated by jLossyWav, if encoded with .flac, is smaller.
I have no idea yet why and if this is good or bad. I hope to get a clearer idea after finishing the source separation.

CODE
jCD1_02.wav.lossy.wav: wrote 995611 bytes, ratio=0,342

CD1_02.wav.lossy.wav: wrote 1026802 bytes, ratio=0,353
Go to the top of the page
+Quote Post
Nick.C
post Nov 30 2008, 20:51
Post #95


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Sounds good [JAZ] - keep up the good work!

The lookup tables, especially in the FFT save a lot of time.

I have used Gregory's "trick" of shifting the lookup index by the appropriate number of bits to reduce (by about half) the memory requirements of some of the lookup tables in what will be lossyWAV 1.1.1f (Delphi).


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Nov 30 2008, 22:00
Post #96





Group: Developer
Posts: 699
Joined: 2-October 08
From: Ottawa
Member No.: 59035



QUOTE (JAZ @ Nov 30 2008, 22:33) *
Some updates on the port to Java:

I have a working code already, based mostly on the C# sources and to a lesser extent to the delphi sources or the web when something didn't look quite right/clear enough.

I don't have yet a .jar that could be tested, but i'll look into making one soon.

Currently:

It is a lot slower than the C# sources. ( by two orders of magnitude). This is because i've removed some of the lookup tables and implemented the FFT methods as they would be by the book.
Of course, i am in the process to add those tables back, in a way that could reflect what they really are and why they are there.

Another thing that i've discovered so far is that while the output seems to produce similar results, it does not produce the same ones than lossyWavSharp.
I've processed a .wav with both programs, and they play fine in foobar, but the one generated by jLossyWav, if encoded with .flac, is smaller.
I have no idea yet why and if this is good or bad. I hope to get a clearer idea after finishing the source separation.

CODE
jCD1_02.wav.lossy.wav: wrote 995611 bytes, ratio=0,342

CD1_02.wav.lossy.wav: wrote 1026802 bytes, ratio=0,353

To my experience, FFT is the most CPU consuming part of the algorithm, practicly the only one that should be really optimized.

At the early stages i also had quite different results from the delphi project. I ended up debugging them simultaniously and comparing internal values at each step. Tedious work, but had to be done. I recently found out that even to this day there are audio files, on which C# version removes one bit more or one bit less on a block or two. I blame the rounding errors, due to the different floating point precision. Delphi used float, double and extended values. C# port only uses float and double. I tried to preserve the float values, even though the temptation was strong to do everything in double. Changing the precision changes the algorithm output. Not that there's something bad about it, the higher the precision - the more accurate the results are, obviously, but in that case i would not be able to compare result to the reference delphi implementation.

I don't really think there can be 100% match, each language does it's arithmetic slightly different when it comes to rounding and such, but the difference in ratio that you quoted (0,342 vs 0,353) indicates that it's not the only issue here right now.

This post has been edited by Gregory S. Chudov: Nov 30 2008, 22:09


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
Nick.C
post Nov 30 2008, 22:04
Post #97


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



The pure Delphi and Delphi / IA-32 / x87 versions of 1.1.1e produce *slightly* different results with my 54 test sample set (less than 512 bytes in 30MBytes of resultant FLAC).


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
[JAZ]
post Dec 1 2008, 23:47
Post #98





Group: Members
Posts: 1783
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



I've uploaded a .jar file with the implementation of the LossyWav algorithm in java.

The file is in this thread:

http://www.hydrogenaudio.org/forums/index....=67697&st=0
Go to the top of the page
+Quote Post
Nick.C
post Dec 2 2008, 08:47
Post #99


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



Thanks again [JAZ] - more potential platforms for lossyWAV processing is certainly a good thing.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Canar
post Dec 9 2008, 05:34
Post #100





Group: Super Moderator
Posts: 3361
Joined: 26-July 02
From: princegeorge.ca
Member No.: 2796



I too would like 32-bit float support, if that's even possible. My reasons are weird: I'm planning on releasing an EP at some point in the not-so-distant future. I want to amplify the release at +15dB past its ReplayGain value to protest the Loudness War. ReplayGain users won't run into the issue, naturally, just everyone else. However, bitrates get kinda crazy with 32-bit float. So I want to use LossyWAV to chop the bitrate down to something sane for release in FLAC.

This post has been edited by Canar: Dec 9 2008, 05:35


--------------------
You cannot ABX the rustling of jimmies.
No mouse? No problem.
Go to the top of the page
+Quote Post

27 Pages V  « < 2 3 4 5 6 > » 
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st September 2014 - 22:59