IPB

Welcome Guest ( Log In | Register )

> Upload forum rules

- No over 30 sec clips of copyrighted music. Cite properly and never more than necessary for the discussion.


- No copyrighted software without permission.


- Click here for complete Hydrogenaudio Terms of Service

50 Pages V  « < 27 28 29 30 31 > »   
Closed TopicStart new topic
lossyWAV Development, WAV bit reduction by 2BDecided
halb27
post Jan 3 2008, 19:08
Post #701





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



Thank you very much for the new version, especially for improving on the bit-to-remove routine and for implementing the correction file mechanism.
Hopefully some more people might find lossyWav attractive because of this feature.

This post has been edited by halb27: Jan 3 2008, 19:12


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Nick.C
post Jan 3 2008, 19:11
Post #702


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (halb27 @ Jan 3 2008, 18:08) *
Thank you very much for the new version.
It's just occurred to me, but the error was in the detection of -ve clipping samples - so conceivably it has adversely influenced recent fine-tuning of the -3 quality preset settings......


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
halb27
post Jan 3 2008, 19:15
Post #703





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Nick.C @ Jan 3 2008, 20:11) *
... so conceivably it has adversely influenced recent fine-tuning of the -3 quality preset settings......

What is your opinion on that? Do you think we can try again to arrive at a lower average bitrate? Or should we be more cautious?


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Nick.C
post Jan 3 2008, 19:51
Post #704


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (halb27 @ Jan 3 2008, 18:15) *
QUOTE (Nick.C @ Jan 3 2008, 20:11) *
... so conceivably it has adversely influenced recent fine-tuning of the -3 quality preset settings......
What is your opinion on that? Do you think we can try again to arrive at a lower average bitrate? Or should we be more cautious?
I think that with the bug allowing undetected -ve clipping "through the net" and not activating the clipping-prevention mechanism when it should have, some samples will have sounded worse than they should have - therefore we could probably be a bit more aggressive.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
halb27
post Jan 3 2008, 20:28
Post #705





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



I see, clipping was not detected on -ve occasion, and thus the clipping prevention strategy was not triggered.

With my personal kind of thinking this is not a sufficient reason for going less defensive, but I see you'ld like to have average bitrate a bit lower, and maybe that's true for other potential users as well.

Well, there is no strict reason to have the -nts defaults where they are right now. I personally am happy with them, but I'm happy too with other defaults especially as we've always wanted to have -nts as a major option.

I think we should encourage users to use other -nts value than the defaults, and the defaults themselves can be like that though better not at the extreme edges.
With -3 for instance I think an -nts usage from -nts 0 to -nts 10 is okay, with -nts 0 playing it very safe, and -nts 10 allowing for potential but very minor audible deviations from the original.
With -1 any -nts value >=0 makes sense, and everybody can choose the security margin he likes.
For -2 any -nts value <= 10 is useful (though a large positive value may be more adequate together with -3).

Maybe we should try to write something like this in the wiki (shall I do it in case I'm allowed to?).


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Nick.C
post Jan 3 2008, 20:32
Post #706


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (halb27 @ Jan 3 2008, 19:28) *
I see, clipping was not detected on -ve occasion, and thus the clipping prevention strategy was not triggered.

With my personal kind of thinking this is not a sufficient reason for going less defensive, but I see you'ld like to have average bitrate a bit lower, and maybe that's true for other potential users as well.

Well, there is no strict reason to have the -nts defaults where they are right now. I personally am happy with them, but I'm happy too with other defaults especially as we've always wanted to have -nts as a major option.

I think we should encourage users to use other -nts value than the defaults, and the defaults themselves can be like that though better not at the extreme edges.
With -3 for instance I think an -nts usage from -nts 0 to -nts 10 is okay, with -nts 0 playing it very safe, and -nts 10 allowing for potential but very minor audible deviations from the original.
With -1 any -nts value >=0 makes sense, and everybody can choose the security margin he likes.
For -2 any -nts value <= 10 is useful (though a large positive value may be more adequate together with -3).

Maybe we should try to write something like this in the wiki (shall I do it in case I'm allowed to?).
I agree with your reasoned response - I think that I was just a bit eager to reduce the bitrate (as usual!).

Feel free to edit the wiki article - you are a major contributor to the project after all....


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
halb27
post Jan 3 2008, 22:07
Post #707





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Nick.C @ Jan 3 2008, 21:32) *
...Feel free to edit the wiki article - you are a major contributor to the project after all....

Done, but can you please look it up and correct errors as I'm not a native English speaker.

BTW, I've moved the player comparison link to the codecs section, and added a remark on Rockbox supporting FLAC and wavPack. Guess the codecs section is the more appropriate place especially as the preset section got a bit fatter by my contribution.


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Nick.C
post Jan 3 2008, 23:07
Post #708


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (halb27 @ Jan 3 2008, 21:07) *
Done, but can you please look it up and correct errors as I'm not a native English speaker.

BTW, I've moved the player comparison link to the codecs section, and added a remark on Rockbox supporting FLAC and wavPack. Guess the codecs section is the more appropriate place especially as the preset section got a bit fatter by my contribution.
Looking good - thanks again for your input and restraining influence....


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Nick.C
post Jan 4 2008, 12:19
Post #709


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Nick.C @ Jan 3 2008, 16:26) *
lossyWAV beta v0.6.1 now appended to post #1 of this thread.
"Fixed" the bug but introduced another (although fairly benign....) expect beta v0.6.2 shortly.

lossyWAV beta v0.6.2 now appended to post #1 of this thread. beta v0.6.1 removed due to bug.

This post has been edited by Nick.C: Jan 4 2008, 12:33


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Nick.C
post Jan 4 2008, 17:07
Post #710


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



I was playing about with FLAC settings on my processed 53 sample set and found the following (all at -b 512):

-0; 45415797B; 3.89s
-1; 46004820B; 4.89s
-2; 44305745B; 4.75s
-3; 41716318B; 4.89s
-4; 42151483B; 5.45s
-5; 40646358B; 8.20s
-6; 40637026B; 9.03s
-7; 40497737B; 29.49s
-8; 40305661B; 38.44s

-3 -m -e -r 2; 40894438B; 13.52s

Is it just me, my sample set or my PC - but does -5 seem to be the sweet spot for encoding?


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
2Bdecided
post Jan 4 2008, 17:19
Post #711


ReplayGain developer


Group: Developer
Posts: 5067
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Is that not the FLAC default? wink.gif

Cheers,
David.
Go to the top of the page
+Quote Post
Nick.C
post Jan 4 2008, 17:23
Post #712


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (2Bdecided @ Jan 4 2008, 16:19) *
Is that not the FLAC default? wink.gif

Cheers,
David.
Erm, yes..... blush.gif Should possibly have stopped to consider why the best time (-5) corresponded to the best time / compression point.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
halb27
post Jan 4 2008, 17:43
Post #713





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Nick.C @ Jan 4 2008, 18:07) *
I was playing about with FLAC settings on my processed 53 sample set and found the following (all at -b 512):

-0; 45415797B; 3.89s
-1; 46004820B; 4.89s
-2; 44305745B; 4.75s
-3; 41716318B; 4.89s
-4; 42151483B; 5.45s
-5; 40646358B; 8.20s
-6; 40637026B; 9.03s
-7; 40497737B; 29.49s
-8; 40305661B; 38.44s

-3 -m -e -r 2; 40894438B; 13.52s

Is it just me, my sample set or my PC - but does -5 seem to be the sweet spot for encoding?

File sizes are plausible, but encoding time is strange, as the internal effort of -5 should be higher compared to -3 -m -e -r 2, at least that's what I learnt from the documentation in case I've read that correctly.
That was the motivation behind my struggling for a fast and space saving setting, and it is like that with my system.
I'm a bit unhappy with your 53 sample set for such tests - in this case because they consist of small snippets to a pretty large extent. Do you mind encoding a small set of full length tracks with -5 and -3 -m -e -r 2?
Other than that -5 is a good FLAC setting of course.


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
2Bdecided
post Jan 4 2008, 17:53
Post #714


ReplayGain developer


Group: Developer
Posts: 5067
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Nick,

This is looking really good. Do you feel you're close to a release candidate yet? It seems that the code works well enough, and it's just a case of settling the command line options and documentation.

Would you be able to find time to explain the algorithm as it currently stands? Or should I just read the code wink.gif Obviously I'm most interested in the changes, but other people might want a full overview without reading 29 pages!

Cheers,
David.
Go to the top of the page
+Quote Post
Nick.C
post Jan 4 2008, 22:31
Post #715


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (2Bdecided @ Jan 4 2008, 16:53) *
Nick,

This is looking really good. Do you feel you're close to a release candidate yet? It seems that the code works well enough, and it's just a case of settling the command line options and documentation.

Would you be able to find time to explain the algorithm as it currently stands? Or should I just read the code wink.gif Obviously I'm most interested in the changes, but other people might want a full overview without reading 29 pages!

Cheers,
David.
Thanks very much David, I appreciate it.

I'm happy with the code - it's fairly robust (now that I've found that bug in the bits-to-remove routine that only caused a crash if I was trying to write a correction file as well...) and the only outstanding item is the reconstitution of the lossy and lwcdf files to recreate the lossless original.

I will try to put the algorithm into engineering speak (my working language) and pass on to you for formalising into audio-processing language.

I should also try to adequately comment the code to allow sense to be made of it by someone other than myself - as it will form part of the release package.

This post has been edited by Nick.C: Jan 4 2008, 22:42


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
jesseg
post Jan 6 2008, 21:49
Post #716





Group: Banned
Posts: 218
Joined: 22-December 02
Member No.: 4194



I had an idea, it might already be this way. I didn't check. But if the correction file was encoded with the bits reversed (16=1/15=2/etc) for whatever bit-depth needed, wouldn't lossless codecs encode that more efficiently? Since it's not really an audio file someone would use directly, that makes sense for starters, and it shouldn't complicate things if lossyWAV-compliant decoding were ever to be built into any lossless decoders. And there should be practically no speed hit at all.


Also, here's a massively improved version of lFLCDrop. smile.gif Hopefully the only thing I'll have to add later is the support for automatic re-assembling of a lossless WAV when the correction file is present. And if anyone has any suggestions about things to add or change with the custom settings area in the batch file - now is the perfect time to suggest something.

QUOTE
lFLCDrop Change Log:
v1.2.0.4
- added correction file settings
- added custom encoding setting

lFLC.bat Change Log:
v1.0.0.4
- added flac decoding support
- added correction file encoding support
- added custom settings section
- improved temp file handling


a newer version is out, check for a later post

This post has been edited by jesseg: Jan 7 2008, 01:54
Go to the top of the page
+Quote Post
carpman
post Jan 6 2008, 22:32
Post #717





Group: Developer
Posts: 1310
Joined: 27-June 07
Member No.: 44789



Hi all,

very nice to see LossyWav development coming on so quickly!

@jesseg

just ran flacdrop v1.2.0.4 with lossywav v.0.6.0.2

settings were

Left=33
Top=550
Encoder=D:\active_encoders_all\lossywav\iFLC.bat
Switches=-2
Image=
OutputDir=
StayOnTop=1
Minimize=1
DeleteSource=0

but unlike when I was using
flacdrop v1.2.0.2 with lossywav v.0.5.4.4 (the last versions I ran) no FLAC files were created. The DOS window came up and it was clearly processing -- I checked the temp directory and could see the lossy.wav files being created but no FLAC files in the Output directory (set as the same as the input directory).

Any ideas?

It could be me but I'm not doing anything differently than with previous version.

ps. I did a search on my hard drive for [inputfilenames].flac but nothing came up.

C.


--------------------
TAK -p4m :: LossyWAV -q 6 | TAK :: Lame 3.98 -V 2
Go to the top of the page
+Quote Post
Nick.C
post Jan 6 2008, 22:58
Post #718


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (jesseg @ Jan 6 2008, 20:49) *
I had an idea, it might already be this way. I didn't check. But if the correction file was encoded with the bits reversed (16=1/15=2/etc) for whatever bit-depth needed, wouldn't lossless codecs encode that more efficiently? Since it's not really an audio file someone would use directly, that makes sense for starters, and it shouldn't complicate things if lossyWAV-compliant decoding were ever to be built into any lossless decoders. And there should be practically no speed hit at all.
Unfortunately, as not all of the differences are positive, this wouldn't help as the new bit 1 of every negative difference would be 1.

Something David said *ages* ago popped back into my head and I want to get a second opinion with respect to my understanding:

Looking for zero values in the sample data was mentioned.

I took this to mean "look for FFT's where all the input values are zero" and have implemented a checking mechanism as follows:

When filling up the FFT input array, OR each sample value with a "running total" which is initialised as zero before the filling starts.

If the resultant "running total" is zero then the FFT is full of zeros and does not require to be calculated.

For every FFT not calculated, do not take a 0db value when calculating the bits_to_remove for the codec_block in question (as rounded zero's are still zero = no added noise).

If *every* FFT was full of zeros, set bits_to_remove to zero and simply store the codec_block.

[edit] This approach reduces my FLAC'd processed 53 sample set by a whole 95 bytes. However, it may slightly increase processing throughput..... [/edit]

This post has been edited by Nick.C: Jan 6 2008, 23:03


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
jesseg
post Jan 6 2008, 23:11
Post #719





Group: Banned
Posts: 218
Joined: 22-December 02
Member No.: 4194



That makes sense. I guess I understand little about how FLAC, and other lossless compression works.


carpman, check your system-drive for the directory that was created of the same path as the files you were trying to encode. You should find the flac files and can delete them. I made a silly booboo and forgot to parse and use the drive letter from the input & output locations.

QUOTE
lFLC.bat Change Log:
v1.0.0.5
- fixed drive letter bug


see attached

This post has been edited by jesseg: Jan 6 2008, 23:11
Go to the top of the page
+Quote Post
halb27
post Jan 6 2008, 23:46
Post #720





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Nick.C @ Jan 6 2008, 23:58) *
... look for FFT's where all the input values are zero ...

The input values to the FFT: isn't that the wave samples of a block?


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
carpman
post Jan 6 2008, 23:57
Post #721





Group: Developer
Posts: 1310
Joined: 27-June 07
Member No.: 44789



QUOTE (jesseg @ Jan 6 2008, 22:11) *
carpman, check your system-drive for the directory that was created of the same path as the files you were trying to encode. You should find the flac files and can delete them.


There weren't any FLAC files created on my system-drive [I searched all drives for the FLAC files and none came up]

Anyway, no matter -- I'll use the new version now -- thanks for the quick reply.

C.


--------------------
TAK -p4m :: LossyWAV -q 6 | TAK :: Lame 3.98 -V 2
Go to the top of the page
+Quote Post
Nick.C
post Jan 7 2008, 00:17
Post #722


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (halb27 @ Jan 6 2008, 22:46) *
QUOTE (Nick.C @ Jan 6 2008, 23:58) *
... look for FFT's where all the input values are zero ...
The input values to the FFT: isn't that the wave samples of a block?
Yes, but at 64 samples and 50% end_overlap and 50% fft_overlap, there are 17 FFT analyses performed per 512 sample codec_block. So, using the existing method, if one whole fft is zero then this brings the whole block's bits_to_remove to zero for 64 samples of silence (which round to silence anyway).....

This post has been edited by Nick.C: Jan 7 2008, 00:20


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
halb27
post Jan 7 2008, 10:28
Post #723





Group: Members
Posts: 2424
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



I see. You look at the codec blocks' subblocks formed by the particular FFTs and ignore the FFT result (not doing the FFT) if the subblock contains silence.
Sounds reasonable though I think for a silent passage only the first and last block (containing partial silence) will take profit of this mechanism as the main silence part affects entire blocks.
I'm not quite sure about the window function. Can the windowing bring FFT input to zero though it's not zero in the wave input? I guess it doesn't but I'm not sure.

This post has been edited by halb27: Jan 7 2008, 10:29


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Nick.C
post Jan 7 2008, 11:06
Post #724


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (halb27 @ Jan 7 2008, 09:28) *
I see. You look at the codec blocks' subblocks formed by the particular FFTs and ignore the FFT result (not doing the FFT) if the subblock contains silence.
Sounds reasonable though I think for a silent passage only the first and last block (containing partial silence) will take profit of this mechanism as the main silence part affects entire blocks.
I'm not quite sure about the window function. Can the windowing bring FFT input to zero though it's not zero in the wave input? I guess it doesn't but I'm not sure.
The silence check is carried out on raw audio samples.

As the checking mechanism is now in place, I am wondering about "what constitutes digital (near) silence"?

I have put in place a few lines of code which take the absolute value of the sample rather than the actual value when performing the silence check.

In determining whether to process a single FFT the threshold for "silence" can be set to zero (total silence) or some value above zero (near silence).

Determining what constitutes "near silence" is the tricky bit......


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Nick.C
post Jan 7 2008, 12:14
Post #725


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



lossyWAV beta v0.6.3 appended to post #1 of this thread.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post

50 Pages V  « < 27 28 29 30 31 > » 
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st August 2014 - 14:09