IPB

Welcome Guest ( Log In | Register )

6 Pages V  « < 2 3 4 5 6 >  
Reply to this topicStart new topic
lossyWAV 1.1.0 released., Added noise WAV bit reduction method.
Nick.C
post Aug 3 2008, 22:13
Post #76


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Hancoque @ Aug 3 2008, 11:59) *
Is that the reason why material with 96 kHz is currently reduced to the same filesize as material with 48 kHz? Are higher sample rates currently treated like if they are played back at a lower speed/rate, thus lowering the frequency spectrum accordingly, which results in a wrong calculation of how many bits can be removed?
Wrong might be a little strong - the sample rate is used correctly where it needs to be but the codec-block-size equates to less time and the granularity of the spreading function will be higher and possibly a bit too coarse.

QUOTE (Hancoque @ Aug 3 2008, 11:59) *
Does that mean that if noise shaping is disabled less bits are removed? That would reassure me.
Exactly the same (except if noise-shaping causes clips) bits-to-remove values will be achieved for noise shaping 1 and 0.

lossyWAV 1.1.0b attached to post #1 in this thread:
  • implementation of increasing fft length for increasing sample rate;
  • improved logfile output and --detail output;
  • reference threshold constants for rectangular dither and triangular dither have been calculated so added noise should be the same for dither off and any dither level between 0 and 1 - the number of bits-to-remove will however reduce with "increasing" dither.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Hancoque
post Aug 4 2008, 03:07
Post #77





Group: Members
Posts: 291
Joined: 27-January 04
From: Germany
Member No.: 11530



Is it intended that the dithering noise is also shaped? A frequency analysis reveals that it's also mildly shaped. I always thought that rectangular or triangular dither corresponds to white noise.

I also wonder if dithering is really that useful. Look at the graph, it shows AC/DC - Fire Your Guns:



Wouldn't you rather use -q 3 without dithering than -q 2.5 with dithering when the bitrate is (nearly) identical?
Go to the top of the page
+Quote Post
Nick.C
post Aug 4 2008, 07:24
Post #78


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Hancoque @ Aug 4 2008, 03:07) *
Is it intended that the dithering noise is also shaped? A frequency analysis reveals that it's also mildly shaped. I always thought that rectangular or triangular dither corresponds to white noise.
Which version of lossyWAV are you using? I ask as the fix to ensure that added noise remains the same when dither is applied has only been introduced at 1.1.0b.

Also, with respect to the triangular dither, I have used a method (discussed on these forums previously) which uses 1 new random number per sample and recycles the previous random number, i.e. tpdf_random = new_random - old_random; (random values in the range +/-0.5). In my implementation and to achieve intermediate dither types I have used: tpdf_random = new_random * dither_type_multiplier - old_random; (dither_type_multiplier in the range 0..1). This may have had an effect on the shape of the dither.

[edit] One other thing: --dither 0 <> no dither. If the dither parameter is used then dither will be applied, somewhere between rectangular and triangular, depending on the parameter value after --dither, i.e. --dither 0 = rectangular dither; --dither 1 = triangular dither; --dither 0.5 = something in between rectangular and triangular dither. [/edit]

This post has been edited by Nick.C: Aug 4 2008, 08:12


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Hancoque
post Aug 4 2008, 11:58
Post #79





Group: Members
Posts: 291
Joined: 27-January 04
From: Germany
Member No.: 11530



I'm using the current version 1.1.0b.

I don't know if it's good to reuse old random values. Why not just use two independent random values? Is the performance impact too high? As it's not used by default I don't think that would be much of an issue.

I have redone the above graph and now differentiated correctly between dither off and dither 0. I also included ReplayGain values to illustrate the theoretically perceived loudness of the noise.
Go to the top of the page
+Quote Post
Nick.C
post Aug 4 2008, 12:36
Post #80


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Hancoque @ Aug 4 2008, 11:58) *
I'm using the current version 1.1.0b.

I don't know if it's good to reuse old random values. Why not just use two independent random values? Is the performance impact too high? As it's not used by default I don't think that would be much of an issue.

I have redone the above graph and now differentiated correctly between dither off and dither 0. I also included ReplayGain values to illustrate the theoretically perceived loudness of the noise.
I'll find my pest control gear and go bug-hunting - the -s 0 -D 1 curve looks suspiciously like it has been noise-shaped.

There is no real performance hit using two independent random values (especially as I have changed from a fibonacci based RNG to a multiply-with-carry RNG) - I'll have a look at that too.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
2Bdecided
post Aug 4 2008, 13:11
Post #81


ReplayGain developer


Group: Developer
Posts: 5063
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Of course the dither should/will be noise shaped. It should/will sit next to the quantisation stage within the noise shaping loop.

Current random number minus previous random number is a well accepted way of getting slightly high passed dither.

Hancoque, no one is suggesting that you should use dither. It's there for people who incorrectly think that they need it - not for those who know that they don't (nor for those that don't know or care!).

Cheers,
David.

This post has been edited by 2Bdecided: Aug 4 2008, 13:12
Go to the top of the page
+Quote Post
Nick.C
post Aug 4 2008, 13:33
Post #82


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (2Bdecided @ Aug 4 2008, 13:11) *
Of course the dither should/will be noise shaped. It should/will sit next to the quantisation stage within the noise shaping loop.

Current random number minus previous random number is a well accepted way of getting slightly high passed dither.

Hancoque, no one is suggesting that you should use dither. It's there for people who incorrectly think that they need it - not for those who know that they don't (nor for those that don't know or care!).

Cheers,
David.
1) Dither is added pre noise shaping;
2) Thanks - it will remain so;
3) I'll leave the option to dither up to the user.

Thanks David.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Hancoque
post Aug 5 2008, 00:29
Post #83





Group: Members
Posts: 291
Joined: 27-January 04
From: Germany
Member No.: 11530



I don't plan to use dithering either. I was just curious. wink.gif

Now that higher sample rates are officially supported, do I understand it right that they require a higher codec block size?

QUOTE (Nick.C @ Jul 28 2008, 21:14) *
I will modify 1.1.0 to increase the FFT lengths at 69.08kHz, 138.15kHz and 276.3kHz, i.e. 64 to 128 to 256 and 512 samples respectively and correspondingly for the other lengths with a similar increase in codec-block length, 512 to 1024 to 2048 to 4096 samples.
So, does this equate to the following sample rate to codec block size analogy?

44.1/48 kHz -> 512
88.2/96 kHz -> 1024
176.4/192 kHz -> 2048
Go to the top of the page
+Quote Post
Gow
post Aug 5 2008, 03:23
Post #84





Group: Members
Posts: 233
Joined: 14-June 06
From: Fort Wayne, IN
Member No.: 31824



I started doing some personal listening and file size tests with 1.1.0b and I must say I am very impressed. The idea behind it all was quite something and now we have it implemented with TAK, FLAC and Wavpack. I use TAK on my computers because that is meant for foobar2000 playback and I use mp3 for my DAP because it is compatible and can be mp3gain'd.

So far, I have been using --extreme and I am in love with the results and it looks like I might switch the all lossless library to the lossyTAK library and if my calculation is correct I should expect the lossyTAK library to be 61% of the size of the current library at -p5. So roughly 305gb down to 186gb and have yet to notice anything that could be a problem sample. Of course, I still have the original image files of the CDs ripped using EAC safely tucked away on DVDs so if I do replace the entire library and a problem is discovered it is quite easy to recover from with the DVD, all done to increase disc longevity and to maintain perfect rips.

Good job, keep it up.

On a theoretical note,

This might move the lossless formats even more into the mainstream although the end file is not technically lossless but it so high quality of an output that the problem samples that exist in the lossy formats do not exist here.

I am all for increasing the overall usage of lossless over lossy and with the lossyWAV a new field of battle has been opened up that might sway those who think lossless is TOO big for use to the lossless side.


--------------------
Zune 80, Tak -p4 audio library, Lossless=Choice
Go to the top of the page
+Quote Post
2Bdecided
post Aug 5 2008, 10:30
Post #85


ReplayGain developer


Group: Developer
Posts: 5063
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (Nick.C @ Aug 4 2008, 13:33) *
1) Dither is added pre noise shaping;
But in-loop or out-of-loop? Traditionally it should be in-loop, like this...

http://www.hydrogenaudio.org/forums/index....st&p=580394

Cheers,
David.
Go to the top of the page
+Quote Post
Nick.C
post Aug 5 2008, 10:57
Post #86


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (2Bdecided @ Aug 5 2008, 10:30) *
But in-loop or out-of-loop? Traditionally it should be in-loop, like this...

http://www.hydrogenaudio.org/forums/index....st&p=580394

Cheers,
David.
Sorry, not clear enough - dither is added in the middle of the noise shaping process, just before rounding.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
smok3
post Aug 7 2008, 09:18
Post #87


A/V Moderator


Group: Moderator
Posts: 1727
Joined: 30-April 02
From: Slovenia
Member No.: 1922



continued, about bat files and tagging, this oneliner seems to be working;

lossyflac.bat
CODE
flac -d %1 --stdout --silent|lossywav - --stdout -P|flac - -b 512 -o %~p1%~n1.lossy.flac --silent && tag --fromfile %1 %~p1%~n1.lossy.flac


requirements: flac, lossywav and tag somewhere on your %path%

example usage: lossyflac file.flac will return file.lossy.flac (with all the tags from file.flac)

p.s. the usual warning about spaces and batch files on win.

edit1: slight fix with paths, added %~p1
edit2: became oneliner

This post has been edited by smok3: Aug 7 2008, 15:10


--------------------
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung
Go to the top of the page
+Quote Post
Nick.C
post Aug 7 2008, 10:12
Post #88


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (smok3 @ Aug 7 2008, 09:18) *
continued, about bat files and tagging, this two liner seems to be working;

lossyflac.bat
CODE
flac -d %1 --stdout --silent|lossywav - --stdout -P|flac - -b 512 -o %~n1.lossy.flac --silent
tag --fromfile %1 %~n1.lossy.flac
...
Very, very elegant - now no need to use foobar2000 for the processing. You could put both lines on one line with a && in between wink.gif.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
smok3
post Aug 7 2008, 13:41
Post #89


A/V Moderator


Group: Moderator
Posts: 1727
Joined: 30-April 02
From: Slovenia
Member No.: 1922



QUOTE
You could put both lines on one line with a && in between

any benefits from that? or do we want oneliner? biggrin.gif

This post has been edited by smok3: Aug 7 2008, 13:44


--------------------
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung
Go to the top of the page
+Quote Post
Nick.C
post Aug 7 2008, 13:53
Post #90


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (smok3 @ Aug 7 2008, 13:41) *
QUOTE
You could put both lines on one line with a && in between
any benefits from that? or do we want oneliner? biggrin.gif
Minimalist, I am a fan of one-line code where it can be done.... The final batch file would be:
CODE
@flac -d %1 --stdout --silent|lossywav - --stdout -P|flac - -b 512 -o %~p1%~n1.lossy.flac --silent && tag --fromfile %1 %~p1%~n1.lossy.flac


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Synthetic Soul
post Aug 7 2008, 14:19
Post #91





Group: Super Moderator
Posts: 4887
Joined: 12-August 04
From: Exeter, UK
Member No.: 16217



One minor benefit: As you are using && (as opposed to &) the call to Tag will only be made if the previous command executed successfully.


--------------------
I'm on a horse.
Go to the top of the page
+Quote Post
smok3
post Aug 7 2008, 15:04
Post #92


A/V Moderator


Group: Moderator
Posts: 1727
Joined: 30-April 02
From: Slovenia
Member No.: 1922



fixed the post - 'downgraded' twoliner to oneliner tongue.gif http://www.hydrogenaudio.org/forums/index....st&p=581437

This post has been edited by smok3: Aug 7 2008, 15:06


--------------------
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung
Go to the top of the page
+Quote Post
Hanky
post Aug 7 2008, 15:16
Post #93





Group: Members (Donating)
Posts: 531
Joined: 18-November 01
From: The Netherlands
Member No.: 481



Nice batch file. But if the specified input file does not exist, this results in a crash of lossywav.exe
Windows XP SP2, lossywav 1.1.0b

CODE
F:\lossywavtest\>flac -d doesnotexist.flac --stdout --silent  | lossy
wav - --stdout -P  | flac - -b 512 -o doesnotexist.lossy.flac --silent

doesnotexist.flac: ERROR initializing decoder
                   init status = FLAC__STREAM_DECODER_INIT_STATUS_ERROR_OPENING_
FlossyWAV 1.1.0b, Copyright (C) 2007,2008 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
ILE

An error occurred opening the input file; it is likely that it does not exist
or is not readable.
ERROR: for encoding a raw file you must specify a value for --endian, --sign, --
channels, --bps, and --sample-rate
Type "flac" for a usage summary or "flac --help" for all options

F:\lossywavtest\>tag --fromfile doesnotexist.flac doesnotexist.lossy.
flac
Tag - Automatic Tag from filename      Copyright (c) 2002-2003 Case
Version 2.0.39b, Compiled 2003-04-11

File not found: 'doesnotexist.lossy.flac'
fromfile: Can't open file 'doesnotexist.flac'.


This post has been edited by Hanky: Aug 7 2008, 15:20
Go to the top of the page
+Quote Post
Nick.C
post Aug 7 2008, 15:48
Post #94


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Hanky @ Aug 7 2008, 15:16) *
Nice batch file. But if the specified input file does not exist, this results in a crash of lossywav.exe
This could probably be dealt with
CODE
@if exist "%1" flac -d "%1" --stdout --silent|lossywav - --stdout -P|flac - -b 512 -o "%~n1.lossy.flac" --silent && tag --fromfile "%1" "%~n1.lossy.flac"


This post has been edited by Nick.C: Aug 7 2008, 15:48


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Hanky
post Aug 7 2008, 15:53
Post #95





Group: Members (Donating)
Posts: 531
Joined: 18-November 01
From: The Netherlands
Member No.: 481



Thanks Nick.C, that fixed the error smile.gif

One more small issue (not lossywav related) is the fact that TAG does not copy embedded cover art.

edit:
I found out I was using an old version of TAG (2.0.39b) and updated to 2.0.52, but that version did not copy embedded cover art either.

But this issue should not be discussed here, rather in Changes Made to TAG.EXE, Tags from text file & OptimFrog support

This post has been edited by Hanky: Aug 7 2008, 16:11
Go to the top of the page
+Quote Post
Nick.C
post Aug 7 2008, 20:03
Post #96


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Hanky @ Aug 7 2008, 15:53) *
Thanks Nick.C, that fixed the error smile.gif
This will process multiple files and place the result in the same directory as the original.
CODE
@echo off
:repeat
if %1.==. goto end
if exist %1 flac -d %1 --stdout --silent|lossywav - --stdout -P --stdinname %1|flac - -b 512 -o "%~dpn1.lossy.flac" --silent && tag --fromfile %1 "%~dpn1.lossy.flac"
shift
goto repeat
:end


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
Hanky
post Aug 7 2008, 20:33
Post #97





Group: Members (Donating)
Posts: 531
Joined: 18-November 01
From: The Netherlands
Member No.: 481



That one works great when you pass one or more filenames, but when I tried with *.flac I got a crash of LossyWav.exe again.
It seems that we need a construction with a FOR (%%I) IN %1 DO loop here to handle wild cards.

This command line does the job:
CODE
FOR %X in (*.flac) DO lossyflac.bat "%X"


edit: quotes added, thanks again Nick.C

This post has been edited by Hanky: Aug 7 2008, 20:48
Go to the top of the page
+Quote Post
smok3
post Aug 7 2008, 20:43
Post #98


A/V Moderator


Group: Moderator
Posts: 1727
Joined: 30-April 02
From: Slovenia
Member No.: 1922



my usual approach is to do one batch that handles one file (action)

and 2nd batch that loops over the first one, (loopy)

one way or the other, depending on the usage. That way you can keep your action bat short and dumb and leave all the file management logic to the loopy one.


--------------------
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung
Go to the top of the page
+Quote Post
Nick.C
post Aug 7 2008, 20:45
Post #99


lossyWAV Developer


Group: Developer
Posts: 1787
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Hanky @ Aug 7 2008, 20:33) *
That one works great when you pass one or more filenames, but when I tried with *.flac I got a crash of LossyWav.exe again.
It seems that we need a construction with a FOR (%%I) IN %1 DO loop here to handle wild cards.

This command line does the job:
CODE
FOR %X in (*.flac) DO lossyflac.bat %X

But this one skips filenames that include spaces.
If you enclose the second %X in double quotes it will handle the files properly.

This post has been edited by Nick.C: Aug 7 2008, 20:45


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
sauvage78
post Aug 17 2008, 06:49
Post #100





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



Quote: Nick.C
"My intention is to understand and implement SebastianG's new noise shaping method, but for that I will also have to introduce / find a PSY model of some kind."

Do you have any clue of how long this will take ? 3 months, 6 months or a year ?
Is SebastianG giving you any accelerated private lessons ?
It's not that I want to hurry you wink.gif or being rude in any way crying.gif , but I care a lot for lossywav ... it's already my favorite lossy codec ... & I plan to convert tera of lossless to Lossy|Tak -P|-p2e ... (without lossless backup) so I care a lot for this new noise shaping method if it can make me save some kbps (& also for the new special Tak setting for lossywav tongue.gif )

without a 1.2.0 development thread I am asking myself everyday:
1: is it a TODO thing that is already actively worked on in the shadows. smile.gif
2: is it a TODO thing that is just an idea. crying.gif
3: are you in vacation with wife & kids. cool.gif
so I'd rather simply ask ...

as I have been disapointed by vaporware feature from Christopher 'Monty' Montgomery in the past ... I am very suspicious about open source developers claims ... so excuse me if I sound rude I just want to test your determination to get lossywav to the max of its possible efficiency. (I don't want the "new noise shaping method" to become the "bitrate peeling" of lossywav)

I know you just released V1.1.0 a month ago & that I shouldn't already be longing for more ... but now that you are a developer you'll have to learn that end-users are relentless vampires !!! LOL

This post has been edited by sauvage78: Aug 17 2008, 07:47


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post

6 Pages V  « < 2 3 4 5 6 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 31st July 2014 - 10:50