IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Something that's been bugging me about FLAC...
TempestGarden
post Dec 25 2011, 21:24
Post #1





Group: Members
Posts: 43
Joined: 12-September 04
From: Philadelphia
Member No.: 16959



So, I have been using FLAC for many, many years now and I have ripped countless CDs and encoded them to FLAC for archive purposes as well as general listening. In all these years, something has been bothering me that I don't understand about this codec and I have decided to post here and see if I can get an answer to this nagging question. Not sure why I waited this long to bring this up, but anyways, here goes...

As you all know, FLAC has various "compression" settings which you can select when encoding the file (which never made sense to me since this is supposed to be a lossless codec). Now I understand that these "compression" levels have been likened to how WinZip compresses a file. I sorta get the analogy but there is something else here that still doesn't make sense to me. To illustrate my question/point better, I did a little experiment and listed my results below. I ripped the same exact song at 4 different "compression" levels and the resulting bitrate that I got for each rip is listed as well.

(Uncompressed) - 1412 kbps
Level 0 - 986 kbps
Level 5 (default) - 933 kbps
Level 8 - 931 kbps

So, based on these results, here is what is bothering me. Between the uncompressed setting and the level 8 setting, there is a loss of almost 500 kbps of data. In my mind, that is quite a substantial amount. I totally understand that whatever setting you chose will ultimately affect the end result bitrate of the file. I get that. What makes no sense to me at all though, is how a file (at level 8 setting) can lose almost 500 kbps of data and still be considered "lossless".

How is that possible? What is in those ~500 kbps worth of data that is getting shaved off? I just don't understand how you can be losing this much data and still have the resulting file be considered lossless. Maybe it's too technical of an answer for me to understand but I figured if anyone would be able to answer this for me, the folks on the forum here would.

So, yeah... this has been nagging at me for awhile. Any help to make me understand this better would be greatly appreciated.

Oh and btw.... Merry Christmas! smile.gif

This post has been edited by TempestGarden: Dec 25 2011, 21:25
Go to the top of the page
+Quote Post
saratoga
post Dec 25 2011, 21:28
Post #2





Group: Members
Posts: 4904
Joined: 2-September 02
Member No.: 3264



QUOTE (TempestGarden @ Dec 25 2011, 15:24) *
What makes no sense to me at all though, is how a file (at level 8 setting) can lose almost 500 kbps of data and still be considered "lossless".


FLAC uses a data compression, which basically stores information more succinctly:

http://en.wikipedia.org/wiki/Data_compression
Go to the top of the page
+Quote Post
benski
post Dec 25 2011, 21:30
Post #3


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



From a layman's point of view, most of the initial compression results from the fact that there is a lot of similarity between the left and right channels and therefore redundant data. For example, instruments that are "center panned" (vocals, kick drum, bass guitar) are duplicated in both channels.
Go to the top of the page
+Quote Post
db1989
post Dec 25 2011, 21:31
Post #4





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



It’s called lossless compression for a reason.

http://en.wikipedia.org/wiki/Lossless_data_compression

Why are you using FLAC at all, never mind for many years, if you have no idea what it’s for? Please do the most basic research before using something and before asking questions like this.

This post has been edited by db1989: Dec 25 2011, 21:36
Go to the top of the page
+Quote Post
TempestGarden
post Dec 25 2011, 22:44
Post #5





Group: Members
Posts: 43
Joined: 12-September 04
From: Philadelphia
Member No.: 16959



Maybe because I didn't fully understand how it worked which was made very clear in my first post. I have been using it THINKING I understood it.

Thanks for the "warm" response btw, I really appreciate it.

This post has been edited by db1989: Dec 27 2011, 01:29
Reason for edit: removing unnecessary full quote
Go to the top of the page
+Quote Post
saratoga
post Dec 25 2011, 23:08
Post #6





Group: Members
Posts: 4904
Joined: 2-September 02
Member No.: 3264



QUOTE (TempestGarden @ Dec 25 2011, 16:44) *
Maybe because I didn't fully understand how it worked which was made very clear in my first post. I have been using it THINKING I understood it.

Thanks for the "warm" response btw, I really appreciate it.


While I'm sure you didn't mean to be rude, if you come and just ask someone to find you information you could easily get from the forums, this website's own FAQ, wikipedia, the flac website, or thousands of other places on google, its relatively inconsiderate of everyone else's time. People will probably be about as considerate back to you.
Go to the top of the page
+Quote Post
psycho
post Dec 25 2011, 23:20
Post #7





Group: Members
Posts: 241
Joined: 14-October 05
Member No.: 25099



TempestGarden, imagine this scenario.

You have a simple text file with the content "I love you, I love you, I love you.". That's 35 characters = 35 bytes. Some very simple form of lossless compression would be to do the folowing (and something like that is happening also with FLAC, plus a lot of more complicated calculaions): The algorithm detects repeating pattern "I love you" and it stores it like this: "I love you"=a, then it detects the comma and does ", "=b and then it stores the data like this:
I love you=a
, =b
ababa.
That's 24 characters = 24 bytes.

You see, none of the information has been lost, because when you decode this, you'll get the exact same file as you had before compression.

I hope this helps.


--------------------
lame -V 0
Go to the top of the page
+Quote Post
TempestGarden
post Dec 25 2011, 23:27
Post #8





Group: Members
Posts: 43
Joined: 12-September 04
From: Philadelphia
Member No.: 16959



Yes, this does help and is a great example! I really appreciate you writing this in an easy to understand example. Thank you very much.

This post has been edited by db1989: Dec 27 2011, 01:28
Reason for edit: removing unnecessary full quote
Go to the top of the page
+Quote Post
Roseval
post Dec 25 2011, 23:41
Post #9





Group: Members
Posts: 476
Joined: 26-March 08
Member No.: 52303



The compression ratio is a source of misunderstanding.
A lot of people thing that it works like the bitrates in MP3 so more or less loss.
The compression ratio simply tells how many CPU FLAC is allowed to use to find the best possible compression (linear prediction).
The more time is allowed the higher the compression.


--------------------
TheWellTemperedComputer.com
Go to the top of the page
+Quote Post
Grinderman
post Dec 25 2011, 23:49
Post #10





Group: Members
Posts: 21
Joined: 30-June 08
Member No.: 55071



QUOTE (TempestGarden @ Dec 25 2011, 21:24) *
So, based on these results, here is what is bothering me. Between the uncompressed setting and the level 8 setting, there is a loss of almost 500 kbps of data. In my mind, that is quite a substantial amount. I totally understand that whatever setting you chose will ultimately affect the end result bitrate of the file. I get that. What makes no sense to me at all though, is how a file (at level 8 setting) can lose almost 500 kbps of data and still be considered "lossless".

Oh and btw.... Merry Christmas! smile.gif

If I'm reading you right, you're not actually struggling with the concept of lossless compression as such, but you feel that "lossless" should always result in exactly the same compressed file size, as nothing is discarded (perhaps not an unreasonable basic assumption on the face of it, regardless of the peevishness and discourtesy you've encountered).

Try thinking of a steel coil spring held between the palms of your outstretched hands - you can make it shorter by squeezing harder, but the more you compress it, the higher the price you pay in terms of effort you have to put in. However, regardless of how much or how little you squeeze it, the substance of the spring remains exactly the same, and in each case it will return to exactly the same state when it's released.

Similarly, with FLAC (or WinZip) you can achieve varying amounts of compression, but higher compression will require more work to be done - you'll need a more powerful CPU to achieve higher compression in a given amount of time, or conversely you'll need more time for a given amount of CPU power. Like the spring, though, all the original data is still there, regardless of the degree to which it's compressed.

It's really about juggling the resources (disk space/time/CPU power) available to you, and choosing the setting which is most appropriate for your individual needs. smile.gif

Go to the top of the page
+Quote Post
TempestGarden
post Dec 26 2011, 00:26
Post #11





Group: Members
Posts: 43
Joined: 12-September 04
From: Philadelphia
Member No.: 16959



^ Another great, easy to understand example. Thank you as well for explaining it to me like this.

QUOTE (Roseval @ Dec 25 2011, 14:41) *
The compression ratio is a source of misunderstanding.
A lot of people thing that it works like the bitrates in MP3 so more or less loss.


Yes, I think you are spot on here. I believe what you said is exactly where the source of my misunderstanding came from.

This post has been edited by db1989: Dec 27 2011, 01:29
Reason for edit: removing unnecessary full quote
Go to the top of the page
+Quote Post
db1989
post Dec 26 2011, 01:07
Post #12





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (TempestGarden @ Dec 25 2011, 21:44) *
Maybe because I didn't fully understand how it worked which was made very clear in my first post. I have been using it THINKING I understood it.

Thanks for the "warm" response btw, I really appreciate it.

I wasnít intending to be rude. I had to ask because I honestly donít understand what you thought FLAC was doing for you if not reducing the size of your tracks (compression) whilst preserving the audio exactly (lossless), and why you used it when you were unaware of its purpose in this way. Iím glad that youíre beginning to understand lossless compression now, but Iím genuinely unsure how you could spend years using it without having gained any idea of its purpose, and (as has been said) it is something for which explanations (such as those you have received in this thread) are readily available, including this from the official site of FLAC:
QUOTE
FLAC stands for Free Lossless Audio Codec, an audio format similar to MP3, but lossless, meaning that audio is compressed in FLAC without any loss in quality. This is similar to how Zip works, except with FLAC you will get much better compression because it is designed specifically for audio
Further information is available on its FAQ.
Go to the top of the page
+Quote Post
mjb2006
post Dec 26 2011, 04:49
Post #13





Group: Members
Posts: 782
Joined: 12-May 06
From: Colorado, USA
Member No.: 30694



I think what the OP needs is just to understand that the official flac encoder's "compression levels" are actually aliases for certain combinations of settings; they are not a single setting being adjusted on a linear scale. From the documentation:

-0 .. -8 = Fastest compression .. highest compression. The default is -5.
-0, --compression-level-0 = Synonymous with -l 0 -b 1152 -r 3
-1, --compression-level-1 = Synonymous with -l 0 -b 1152 -M -r 3
-2, --compression-level-2 = Synonymous with -l 0 -b 1152 -m -r 3
-3, --compression-level-3 = Synonymous with -l 6 -b 4096 -r 4
-4, --compression-level-4 = Synonymous with -l 8 -b 4096 -M -r 4
-5, --compression-level-5 = Synonymous with -l 8 -b 4096 -m -r 5
-6, --compression-level-6 = Synonymous with -l 8 -b 4096 -m -r 6
-7, --compression-level-7 = Synonymous with -l 8 -b 4096 -m -e -r 6
-8, --compression-level-8 = Synonymous with -l 12 -b 4096 -m -e -r 6

The meaning of -l, -b, etc. can be found in the docs.
Go to the top of the page
+Quote Post
Speckmade
post Dec 26 2011, 06:04
Post #14





Group: Members
Posts: 36
Joined: 15-February 05
Member No.: 19848



QUOTE (psycho @ Dec 25 2011, 23:20) *
I love you=a
, =b
ababa.
That's 24 characters = 24 bytes.

Now this is an example of removing one type of so-called redundancy that isn't actually used with lossless audio compression AFAIK.
You might find it fascinating to find out not only about the already mentioned (stereo) channel decoupling, but also the prediction that takes advantage of the correlation between subsequent samples, and entropy coding that deals with values that appear more or less often (as an effect of the decorrelation).
Although, if you're unfamiliar with pulse-code modulation (PCM) you might wanna find out about that first...
Go to the top of the page
+Quote Post
saratoga
post Dec 26 2011, 07:34
Post #15





Group: Members
Posts: 4904
Joined: 2-September 02
Member No.: 3264



That is a simple form of entropy coding, which is used in FLAC.

This post has been edited by db1989: Dec 27 2011, 01:29
Reason for edit: removing unnecessary full quote
Go to the top of the page
+Quote Post
slks
post Dec 26 2011, 10:54
Post #16





Group: Members
Posts: 392
Joined: 31-March 06
From: Houston, Texas
Member No.: 29046



To paraphrase what's already been said -

The different FLAC level settings (-0 through -8) just spend more computing power to search for increasingly more efficient ways to pack the same data. The higher the setting, the more CPU cycles (time) will be used to compress the file to as little bits as possible (smaller filesize).

If you are still concerned, you could compare your original, uncompressed .wav file to FLACs that have been encoded and decoded at various compression levels. Going through them, bit by bit, you will see that they are identical. foobar2000 has a plugin to do this easily, I believe it was called something like "Bit-compare".


--------------------
http://www.last.fm/user/sls/
Go to the top of the page
+Quote Post
halb27
post Dec 26 2011, 11:34
Post #17





Group: Members
Posts: 2425
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



A simple model for lossless audio compression:
Instead of describing each wave sample as a 16 bit integer as is done with wav files, it can be describred as the difference to the previous wave sample. As the difference is a small value usually it can usually be described with far less than 16 bit. All it takes is an efficient and flexible coding scheme for encoding the difference values.
Real life lossless codecs like FLAC are more sophisticated taking in account more than 1 of the previous samples and the correlelation between left and riigh channel as was said before.

This post has been edited by halb27: Dec 26 2011, 11:43


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
Ron Jones
post Dec 26 2011, 19:03
Post #18





Group: Members
Posts: 412
Joined: 9-August 07
From: Los Angeles
Member No.: 46048



QUOTE (Grinderman @ Dec 25 2011, 15:49) *
Try thinking of a steel coil spring held between the palms of your outstretched hands - you can make it shorter by squeezing harder, but the more you compress it, the higher the price you pay in terms of effort you have to put in. However, regardless of how much or how little you squeeze it, the substance of the spring remains exactly the same, and in each case it will return to exactly the same state when it's released.

That's a fantastic analogy. I'll have to remember that one.

QUOTE (db1989 @ Dec 25 2011, 17:07) *
I wasnít intending to be rude. I had to ask because I honestly donít understand what you thought FLAC was doing for you if not reducing the size of your tracks (compression) whilst preserving the audio exactly (lossless), and why you used it when you were unaware of its purpose in this way.

He was aware of this, as was stated in his first post:
QUOTE
I just don't understand how you can be losing this much data and still have the resulting file be considered lossless.
Go to the top of the page
+Quote Post
Speckmade
post Dec 26 2011, 21:09
Post #19





Group: Members
Posts: 36
Joined: 15-February 05
Member No.: 19848



QUOTE (saratoga @ Dec 26 2011, 07:34) *
QUOTE (Speckmade @ Dec 26 2011, 00:04) *
QUOTE (psycho @ Dec 25 2011, 23:20) *

ababa

...isn't actually used...

That is a simple form of entropy coding, which is used in FLAC.

I don't understand - this doesn't sound right.
This is a dictionary coder. It replaces strings of arbitrary length with place holders whereas entropy coding replaces (fixed-lenght) symbols. FLAC uses Rice coding and Run-length encoding.
Go to the top of the page
+Quote Post
TBeck
post Dec 27 2011, 00:15
Post #20


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



I have to agree.

I don't know of any lossless audio compressor utilizing something like this.

Well, ok, once i read LPAC is coding two consecutive zeros as one bit, because no appropriate rice code for very low volume signals (consisting mostly of zeros) exists. But this is a very specific or unique way to deal with quite rare signals.

This post has been edited by db1989: Dec 27 2011, 00:51
Reason for edit: removing unnecessary full quote
Go to the top of the page
+Quote Post
Speckmade
post Dec 27 2011, 03:50
Post #21





Group: Members
Posts: 36
Joined: 15-February 05
Member No.: 19848



QUOTE (TBeck @ Dec 27 2011, 00:15) *
no appropriate rice code for very low volume signals

FLAC uses RLE for that, as I understand.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd August 2014 - 11:50