IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
LossyFLAC yielding higher bitrate than FLAC alone
Hex144
post May 9 2013, 16:17
Post #1





Group: Members
Posts: 33
Joined: 13-February 13
Member No.: 106615



Hi

I'm not aware of this issue being known so here it goes...
I noticed that LossyWAV + FLAC (with flac block size of 512) may produce larger files than FLAC alone for some simple music material that already encodes at low bitrates with FLAC.

tools used: lossyWAV 1.3.0, FLAC 1.2.1
example bitrates for a track which consists of solo piano (44.1 kHz, 16 bit, stereo):

CODE
flac -8     375
-q E, -8    393
-q S, -8    390

flac -5     379
-q E, -5    395
-q S, -5    393


Any thoughts?

This post has been edited by Hex144: May 9 2013, 16:52
Go to the top of the page
+Quote Post
Dynamic
post May 9 2013, 16:53
Post #2





Group: Members
Posts: 832
Joined: 17-September 06
Member No.: 35307



This is known in the development threads, particularly with clean recordings of simple classical and solo tonal instruments at modest volume levels. It is a result of the shorter block length (512 samples versus 4096 for FLAC which reduces the lossless encoder's efficiency a little) and the relative lack of bits removed by the lossy process compared to loud, spectrally-dense materials (partly because of the volume level, partly because there may be areas of the spectrum with no content, causing lossyWAV to assume there's a very low noise floor and no scope to remove bits.

It's possible that the more aggressive settings such as -C (eConomic) -P (portable) and -X (eXtraportable) would still offer a reduction, especially on the most recent versions that employ adaptive noise shaping to increase the bit-depth reduction and reduce bitrate (-X reaching around 310 kbps typically).

It's not something we worry about unduly as the files are small in any case and the loss of compression on these is far, far smaller than the compression advantage gained on more typical music. This effect doesn't seem to be observed unless the lossless FLAC bitrate is less than about 500 kbps or so.
Go to the top of the page
+Quote Post
Hex144
post May 9 2013, 18:21
Post #3





Group: Members
Posts: 33
Joined: 13-February 13
Member No.: 106615



So what I'm getting is that it's not really LossyWAV's "fault", but FLAC's, with its small block size.
I'm more interested in the higher quality presets, for archiving music, and this thing seems to happen on many of the quieter tracks (for example, using 'Extreme', a Chopin nocturne can grow by 7% while probably not being bit-for-bit lossless). Guess I'll have to be careful.

Thanks for replying.

This post has been edited by Hex144: May 9 2013, 18:23
Go to the top of the page
+Quote Post
carpman
post May 9 2013, 20:40
Post #4





Group: Developer
Posts: 1342
Joined: 27-June 07
Member No.: 44789



Might be useful:
http://www.hydrogenaudio.org/forums/index....st&p=718766

My view:
QUOTE (carpman @ Aug 17 2010, 17:13) *
Personally on solo piano music, I skip lossyWAV (TAK on its [own] performs very well, approx 350 - 550 kbps).

C.


--------------------
TAK -p4m :: LossyWAV -q 6 | TAK :: Lame 3.98 -V 2
Go to the top of the page
+Quote Post
halb27
post May 9 2013, 20:43
Post #5





Group: Members
Posts: 2446
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



The small FLAC blocksize is a requirement which originates from lossyWAV in order to make lossyWAV useful.
For quiet classical music and very high lossyWAV quality settings the situation described may occur not very seldom.
But you can always compare the file size lossyWAV | FLAC and FLAC, and use the more appropriate files, or decide a priori according to the musical contents as carpman said.

This post has been edited by halb27: May 9 2013, 20:45


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Hex144
post May 10 2013, 12:22
Post #6





Group: Members
Posts: 33
Joined: 13-February 13
Member No.: 106615



Thanks Halb and Carpman.
Go to the top of the page
+Quote Post
2Bdecided
post May 10 2013, 14:24
Post #7


ReplayGain developer


Group: Developer
Posts: 5362
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (halb27 @ May 9 2013, 20:43) *
The small FLAC blocksize is a requirement which originates from lossyWAV in order to make lossyWAV useful.
It's needed to make lossyWAV more efficient, but it still works with a 4096-sample blocklength - and in these rare cases, works better. Whether it works well enough to bother with lossyWAV at all depends on the source material. A solo piano recording with significant background noise (e.g. an old analogue recording) might benefit considerably. A pristine digital recording with low noise would benefit less. It is theoretically possible, even with the same blocksize, for lossyWAV to increase the bitrate - but I've yet to see it.

Even without lossyWAV, FLAC itself could benefit a little from adjusting its blocksize for optimal efficiency depending on the input signal, but no FLAC encoder bothers. FLAC decoders are supposed to handle it.

Cheers,
David.

Go to the top of the page
+Quote Post
jensend
post May 10 2013, 20:16
Post #8





Group: Members
Posts: 149
Joined: 21-May 05
Member No.: 22191



As a spectator, early on in LossyWAV's development I wondered about the 512 blocksize. Does the noise floor really change that much from one 10ms period to the next? Trawling through the archives I see 2BDecided started off with 4096 sample blocks because it was flac's default, then switched to 1024 sample blocks when that looked substantially better in his testing, but I wonder how extensive the testing was that fixed on 512.*

Might the optimal lossywav blocksize depend on musical content/genre and on lossywav aggressiveness? I think so. With the current lossywav, some classical music I had on hand compressed better with flac -b 512 when using lossywav 'high' quality but did better with -b 1024 when using lossywav 'extreme'. (In both cases the difference was <1% though.)

4096 samples is understandably too big- and regular FLAC doesn't benefit much from it either. Most of the benefit flac can get from >512 sample blocks is already achieved at 1024 (similarly, 512-sample blocks already eliminate most of the overhead from 256-sample blocks).

*Here's the only stuff I found about that in a quick search: this post refers to a post that's not in its thread anymore- maybe this one?
Go to the top of the page
+Quote Post
Hex144
post May 10 2013, 20:47
Post #9





Group: Members
Posts: 33
Joined: 13-February 13
Member No.: 106615



QUOTE (2Bdecided @ May 10 2013, 16:24) *
QUOTE (halb27 @ May 9 2013, 20:43) *
The small FLAC blocksize is a requirement which originates from lossyWAV in order to make lossyWAV useful.
It's needed to make lossyWAV more efficient, but it still works with a 4096-sample blocklength - and in these rare cases, works better.

Thinking about it, it makes sense since lossyWAV does little to no pre-processing. So encoding a few "problem" files using flac's default blocksize led to the same bitrate as the flac original (no big surprise)...
Go to the top of the page
+Quote Post
Dynamic
post May 12 2013, 09:34
Post #10





Group: Members
Posts: 832
Joined: 17-September 06
Member No.: 35307



On more thing - I seem to recall that TAK introduced features for improving lossyTAK compression by noticing when the bit-depth stayed the same for consecutive blocks of 512 and increasing the block size for those blocks adaptively, so TAK might not show the same 7% increase. TAK tends to compress a little bit more than FLAC anyway.

(I may be misremembering - it was only from a glance through TAK changelogs or HA forum threads - I never tried it)

The problem lies then with compatibility - TAK is not widely supported, while FLAC is.

I guess, if anyone felt it worthwhile, FLAC encoders could implement an adaptive block-size mode that looks at bit-depth in 512-sample chunks and reduces the block size to suit the consecutive blocks of equal bit-depth, while reverting to 4096 (or the set blocksize) in other cases. This would retain 4096-sample blocks when possible and handle lossyWAV efficiently with relatively little processor overhead. The only potential problem is if we discovered any decoders that fail to handle changing block size properly.

This post has been edited by Dynamic: May 12 2013, 09:35
Go to the top of the page
+Quote Post
lvqcl
post May 12 2013, 11:18
Post #11





Group: Developer
Posts: 3467
Joined: 2-December 07
Member No.: 49183



There's also --merge-blocks option in Wavpack.
Go to the top of the page
+Quote Post
TBeck
post May 12 2013, 12:31
Post #12


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



QUOTE (Dynamic @ May 12 2013, 10:34) *
On more thing - I seem to recall that TAK introduced features for improving lossyTAK compression by noticing when the bit-depth stayed the same for consecutive blocks of 512 and increasing the block size for those blocks adaptively, so TAK might not show the same 7% increase. TAK tends to compress a little bit more than FLAC anyway.

(I may be misremembering - it was only from a glance through TAK changelogs or HA forum threads - I never tried it)

Early beta releases of TAK 2.1.0 contained a dedicated LossyWav-Codec (basically a variation of the main codec). Advantages:
  1. About 2 percent better compression for my test corpus.
  2. Usually not more than 0.1 percent compression penality for the afore-noted problem files.

This codec had been removed because there seemed to be little interest.

Maybe i will reintegrate it sometime.
Go to the top of the page
+Quote Post
Dynamic
post May 12 2013, 17:04
Post #13





Group: Members
Posts: 832
Joined: 17-September 06
Member No.: 35307



Thanks, TBeck. I guess the demand is going to be limited what with lossyWAV's modest use and TAK's limited device support compared to FLAC (and Wavpack which is in Rockbox) so lumbering people with extra features for a small userbase isn't terribly worthwhile unless lossyTAK becomes more widely used.

Thanks, lvqcl. Using Wavpack with --merge-blocks option may be a better option than FLAC for a user with a Rockbox DAP and this type of content. I noticed the switch was absent from one example in the lossyWAV Wiki (now edited) but was present in the fb2k and EAC lossyWV examples, so I'm glad you mentioned it here.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd December 2014 - 03:47