IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Is a process really lossless?
evereux
post Aug 4 2005, 09:44
Post #1





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



Since there seems to be a steady stream of people claiming that a lossless process isn't lossless I've decided to write a mini guide showing how the user can themselves check to see if a process was indeed lossless. Hopefully this will help some gain a better understanding of what lossless is.

For the purposes of this mini tutorial I'll be compressing a file using Wavpack. You can use this methodology for any lossless process, you just need to know how to decompress that file back to wav.

The guide below will be specific to Windows OS's but the principles are true for any other OS. The text within the code boxes are copied from my DOS prompt. To access your command prompt go to Start > Run > Cmd.


Create a new folder in root called lossless and put in this folder your wav file.
CODE
C:\lossless>dir
Volume in drive C has no label.
Volume Serial Number is AC06-24BF

Directory of C:\lossless

04/08/2005  09:20    <DIR>          .
04/08/2005  09:20    <DIR>          ..
04/08/2005  09:19        50,862,044 01._Neighborhood_#1_(Tunnels).wav
              1 File(s)     50,862,044 bytes
              2 Dir(s)   1,017,434,112 bytes free

C:\lossless>



Compress your wav file with your lossless compressor.
CODE
C:\lossless>wavpack -h 01._Neighborhood_#1_(Tunnels).wav

WAVPACK  Hybrid Lossless Wavefile Compressor  Win32 Version 4.2  2005-04-02
Copyright (c) 1998 - 2005 Conifer Software.  All Rights Reserved.

created 01._Neighborhood_#1_(Tunnels).wv in 12.38 secs (lossless, 33.34%)

C:\lossless>



Now we have our compressed file you'll need to rename this file so that when we come to decompress the file it will not over write the original file.
CODE
C:\lossless>dir
Volume in drive C has no label.
Volume Serial Number is AC06-24BF

Directory of C:\lossless

04/08/2005  09:26    <DIR>          .
04/08/2005  09:26    <DIR>          ..
04/08/2005  09:19        50,862,044 01._Neighborhood_#1_(Tunnels).wav
04/08/2005  09:22        33,905,312 01._Neighborhood_#1_(Tunnels)[new].wv
              2 File(s)     84,767,356 bytes
              2 Dir(s)     983,523,328 bytes free

C:\lossless>



Next we will decode the .wv file.
CODE
C:\lossless>wvunpack "C:\lossless\01. Neighborhood #1 (Tunnels)[new].wv"

WVUNPACK  Hybrid Lossless Wavefile Decompressor  Win32 Version 4.2  2005-04-02
Copyright (c) 1998 - 2005 Conifer Software.  All Rights Reserved.

file C:\lossless\01. Neighborhood #1 (Tunnels)[new].wv not found!

C:\lossless>wvunpack "01._Neighborhood_#1_(Tunnels)[new].wv"

WVUNPACK  Hybrid Lossless Wavefile Decompressor  Win32 Version 4.2  2005-04-02
Copyright (c) 1998 - 2005 Conifer Software.  All Rights Reserved.

restored 01._Neighborhood_#1_(Tunnels)[new].wav in 8.46 secs (lossless, 33.34%)

C:\lossless>

You should now have the new decompressed file in your folder.


Now comes the essential part in being able to check if a process is lossless. We do this by running a binary file compare using Windows' FC function. What this does is compare two files and if there are any differences they will be printed to the screen.
CODE
C:\lossless>FC /B C:\lossless\01._Neighborhood_#1_(Tunnels).wav C:\lossless\01._Neighborhood_#1_(Tunnels)[new].wav
Comparing files C:\LOSSLESS\01._Neighborhood_#1_(Tunnels).wav and C:\LOSSLESS\01
._NEIGHBORHOOD_#1_(TUNNELS)[NEW].WAV
FC: no differences encountered


C:\lossless>

As you can see, the above process was perfectly free from any kind of loss and we can be confident this is the case since FC reports no differences encountered.

edit: command changed to FC /B, thanks zZzZzZz

This post has been edited by evereux: Aug 4 2005, 10:03


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
evereux
post Aug 4 2005, 09:46
Post #2





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



There are times when using FC /B file1.ext file2.ext from the command prompt will yield different results after compressing a file with a lossless compressor.

There is a very good chance that the reason for this is that the original wav file contained some additional header informations (not related to the audio stream). Once the file is losslessly compressed and then decompressed these header informations are changed or lost.

We can use a tool like Foobar2000's bitcompare to just check the audio stream. To do this you will need to download and install Foobar2000.

Load your 2 wav files into Foobar2000 by simply dragging and dropping the files into Foobar2000's main window.

Highlight your two files by doing CTRL+A.

Right click whilst your files are highlighted and select Bit Compare. See image > foo_bitcompare

Once the comparison is finished the console window will open and you should be presented with the following information. See image > foo_bitcompare results
As you can see, the console window reports:
INFO (foo_bitcompare) : No differences in decoded data found.
INFO (foo_bitcompare) : Finished successfully.

This post has been edited by evereux: Aug 4 2005, 11:17


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
Peter
post Aug 4 2005, 09:48
Post #3


foobar2000 developer


Group: Admin
Posts: 3275
Joined: 30-September 01
Member No.: 84



I believe the commandline should be FC /B rather than just FC, otherwise it compares the files in text mode.
Go to the top of the page
+Quote Post
archagon
post Aug 4 2005, 09:51
Post #4





Group: Members
Posts: 110
Joined: 27-February 05
From: Silicon Valley
Member No.: 20184



You LIE! It's all an Illuminati conspiracy!
Go to the top of the page
+Quote Post
kl33per
post Aug 4 2005, 09:55
Post #5


A/V Moderator


Group: Members
Posts: 841
Joined: 9-June 03
From: Brisbane, AUS
Member No.: 7078



What if the WAV headers change. Won't this lead to file's not comparing acurately, even though the audio is exactly the same.


--------------------
www.sessions.com.au - Sessions Entertainment
Go to the top of the page
+Quote Post
Defsac
post Aug 4 2005, 10:00
Post #6





Group: Members
Posts: 347
Joined: 17-May 05
Member No.: 22107



QUOTE (kl33per @ Aug 4 2005, 06:55 PM)
What if the WAV headers change.  Won't this lead to file's not comparing acurately, even though the audio is exactly the same.
*
Yes. foobar2000's bitcompare would probably be a better option because I believe it only compares actual audio data.
Go to the top of the page
+Quote Post
pest
post Aug 4 2005, 10:01
Post #7





Group: Members
Posts: 208
Joined: 12-March 04
From: Germany
Member No.: 12686



QUOTE (kl33per @ Aug 4 2005, 12:55 AM)
What if the WAV headers change.  Won't this lead to file's not comparing acurately, even though the audio is exactly the same.


wavpack stores the complete waveheader and additional chunks...flac doesn't
so this might not work with partially broken wave-files...

'fc /b' is the right command
Go to the top of the page
+Quote Post
PoisonDan
post Aug 4 2005, 10:05
Post #8





Group: Members (Donating)
Posts: 678
Joined: 10-December 01
From: Belgium
Member No.: 622



QUOTE (kl33per @ Aug 4 2005, 10:55 AM)
What if the WAV headers change.  Won't this lead to file's not comparing acurately, even though the audio is exactly the same.
*

True. For a more reliable comparison, something like foobar2000's bitcompare function should be used. The problem is, when you mention this in the guide you're expecting the user to install foobar2000, while FC is readily available from the OS.

Related to this, do you know of a command-line tool that works similar to foobar2000's bitcompare (i.e. only compare audio data)?


--------------------
Over thinking, over analyzing separates the body from the mind.
Go to the top of the page
+Quote Post
Defsac
post Aug 4 2005, 10:08
Post #9





Group: Members
Posts: 347
Joined: 17-May 05
Member No.: 22107



QUOTE (PoisonDan @ Aug 4 2005, 07:05 PM)
Related to this, do you know of a command-line tool that works similar to foobar2000's bitcompare (i.e. only compare audio data)?
*
WavPack will compute and verify MD5 sums during compression and decompression if you use the '-m' argument, you don't need to use an external tool for verification.

This post has been edited by Defsac: Aug 4 2005, 10:09
Go to the top of the page
+Quote Post
bug80
post Aug 4 2005, 10:34
Post #10





Group: Members
Posts: 398
Joined: 23-January 05
From: The Netherlands
Member No.: 19254



Hm, interesting. First, let me state that I believe that lossless = lossless, of course.

However, this kind of evidence is a bit tricky. You use one file, which is encoded lossless, to prove your statement. But, how do you prove it works for all files?

I would say, lossless compression is really lossless, untill someone comes up with a sample on which it doesn't work (innocent untill proven otherwise). smile.gif

This post has been edited by bug80: Aug 4 2005, 10:34
Go to the top of the page
+Quote Post
PoisonDan
post Aug 4 2005, 10:38
Post #11





Group: Members (Donating)
Posts: 678
Joined: 10-December 01
From: Belgium
Member No.: 622



QUOTE (bug80 @ Aug 4 2005, 11:34 AM)
However, this kind of evidence is a bit tricky. You use one file, which is encoded lossless, to prove your statement. But, how do you prove it works for all files?
*

You can't. That also why evereux specifically wrote:
QUOTE
a mini guide showing how the user can themselves check to see if a process was indeed lossless.

You can prove it for one particular process, but not for everything.


--------------------
Over thinking, over analyzing separates the body from the mind.
Go to the top of the page
+Quote Post
Sunhillow
post Aug 4 2005, 10:41
Post #12





Group: Members (Donating)
Posts: 483
Joined: 13-October 01
From: Stuttgart
Member No.: 286



QUOTE (bug80 @ Aug 4 2005, 11:34 AM)
However, this kind of evidence is a bit tricky. You use one file, which is encoded lossless, to prove your statement. But, how do you prove it works for all files?
*


This would be a bug in the compression algorithm - just like if a .exe had differences after packing and unpacking process.
Go to the top of the page
+Quote Post
evereux
post Aug 4 2005, 10:41
Post #13





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE (PoisonDan @ Aug 4 2005, 09:05 AM)
QUOTE (kl33per @ Aug 4 2005, 10:55 AM)
What if the WAV headers change.  Won't this lead to file's not comparing acurately, even though the audio is exactly the same.
*

True. For a more reliable comparison, something like foobar2000's bitcompare function should be used. The problem is, when you mention this in the guide you're expecting the user to install foobar2000, while FC is readily available from the OS.
*


This is why I've used FC for the guide above. If one wav file contains header information what will be outputted to the terminal screen using FC? I'm sure it would be quite obvious that it's only the header data that's different?

QUOTE (Defsac @ Aug 4 2005, 09:08 AM)
QUOTE (PoisonDan @ Aug 4 2005, 07:05 PM)
Related to this, do you know of a command-line tool that works similar to foobar2000's bitcompare (i.e. only compare audio data)?
*
WavPack will compute and verify MD5 sums during compression and decompression if you use the '-m' argument, you don't need to use an external tool for verification.
*


My example uses wavpack but it wasn't my intention for this process to be specific to wavpack. The question PoisonDan asks should be answered in a manner applicable to all lossless processes.

QUOTE (bug80 @ Aug 4 2005, 09:34 AM)
Hm, interesting. First, let me state that I believe that lossless = lossless, of course.

However, this kind of evidence is a bit tricky. You use one file, which is encoded lossless, to prove your statement. But, how do you prove it works for all files?
*

It's up to someone to show us otherwise instead of blindly stating so. My first post here helps the user to do that.


I'll edit my second post to include Foobar2000s bit compare feature sometime soon.


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
kjoonlee
post Aug 4 2005, 10:42
Post #14





Group: Members
Posts: 2526
Joined: 25-July 02
From: South Korea
Member No.: 2782



Lossless compression is a mathematical process.

A mathematical proof would be possible, I'm sure.


--------------------
http://blacksun.ivyro.net/vorbis/vorbisfaq.htm
Go to the top of the page
+Quote Post
A_Man_Eating_Duc...
post Aug 4 2005, 10:45
Post #15





Group: Members
Posts: 925
Joined: 21-December 01
From: New Zealand
Member No.: 705



Could we get this stickied?

This post has been edited by A_Man_Eating_Duck: Aug 4 2005, 10:46


--------------------
Who are you and how did you get in here ?
I'm a locksmith, I'm a locksmith.
Go to the top of the page
+Quote Post
pest
post Aug 4 2005, 10:51
Post #16





Group: Members
Posts: 208
Joined: 12-March 04
From: Germany
Member No.: 12686



QUOTE (A_Man_Eating_Duck @ Aug 4 2005, 01:45 AM)
Could we get this stickied?


yeah for the unbelievers tongue.gif
Go to the top of the page
+Quote Post
Acid8000
post Aug 4 2005, 11:47
Post #17





Group: Members
Posts: 279
Joined: 14-May 05
From: Sydney
Member No.: 22048



http://www.hydrogenaudio.org/forums/index....showtopic=36043

This somehow gave me an idea. How about, making a .png of a moir pattern, converting to wav, compressing losslessly, then doing the reverse and seeing if it looks the same.


--------------------
Acid8000 aka. PhilDEE
Go to the top of the page
+Quote Post
TCM
post Aug 4 2005, 13:55
Post #18





Group: Members
Posts: 87
Joined: 14-July 05
Member No.: 23325



QUOTE (Acid8000 @ Aug 4 2005, 12:47 PM)
How about, making a .png of a moir pattern, converting to wav, compressing losslessly, then doing the reverse and seeing if it looks the same.


hmm, what's the point? most people don't normally convert their music to png.

in a process like png -> wav -> flac/wv/... -> wav -> png where the two wavs are identical, what additional proof would the png bring? of course, this is ignoring the fact that you assume correct operation of the png compression.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th July 2014 - 05:35