IPB

Welcome Guest ( Log In | Register )

2 Pages V  < 1 2  
Reply to this topicStart new topic
Any encoders which will 'accept but crop' too high resolution , ... so that the user will notice that not all formats are equal?
saratoga
post Jan 4 2011, 19:55
Post #26





Group: Members
Posts: 5161
Joined: 2-September 02
Member No.: 3264



QUOTE (2Bdecided @ Jan 4 2011, 07:02) *
QUOTE (saratoga @ Jan 4 2011, 04:06) *
Floating point is only approximate
It's a digital file. In which way is a copy of that file "approximate"?


The ones and zeros making up a floating point file do not necessarily have a consistent meaning across different CPUs and even operating systems, and the output of identical floating point operations on different systems is not required (or even expected) to be identical. The values represented in each floating point sample are therefore literally approximations of an (unknown) true value that is not recorded.

QUOTE (2Bdecided @ Jan 4 2011, 07:02) *
What do you think the words "copy" and "lossless" mean in the context of binary data?


I think lossless means you should be able to save a value and then load it latter in a machine independent manner with a guarantee that you will get an identical value out latter. I'm skeptical that this is possible for a user mode program like flac short of running it in a virtual machine or reference floating point emulator. I think the simple process of converting between different machine representations of floating point values is likely to be lossy. Although someone more familiar with IEEE754 compliance may know better then I.
Go to the top of the page
+Quote Post
googlebot
post Jan 4 2011, 20:04
Post #27





Group: Members
Posts: 698
Joined: 6-March 10
Member No.: 78779



AFAIK the meaning is well defined. Just the precision of operations may differ from machine to machine. But this is only an issue when you want to exploit a specific machine's performance benefits regarding FP calculation. Then you need to know your minimum requirement. Limited hardware doesn't prevent FP math of arbitrary precision, though. It's all a question of effort.

This post has been edited by googlebot: Jan 4 2011, 20:14
Go to the top of the page
+Quote Post
lvqcl
post Jan 4 2011, 20:22
Post #28





Group: Developer
Posts: 3468
Joined: 2-December 07
Member No.: 49183



QUOTE (saratoga @ Jan 4 2011, 21:55) *
I think lossless means you should be able to save a value and then load it latter in a machine independent manner with a guarantee that you will get an identical value out latter. I'm skeptical that this is possible for a user mode program like flac short of running it in a virtual machine or reference floating point emulator.

It is definitely possible for 7zip or winrar.
Go to the top of the page
+Quote Post
saratoga
post Jan 4 2011, 20:39
Post #29





Group: Members
Posts: 5161
Joined: 2-September 02
Member No.: 3264



QUOTE (lvqcl @ Jan 4 2011, 14:22) *
QUOTE (saratoga @ Jan 4 2011, 21:55) *
I think lossless means you should be able to save a value and then load it latter in a machine independent manner with a guarantee that you will get an identical value out latter. I'm skeptical that this is possible for a user mode program like flac short of running it in a virtual machine or reference floating point emulator.

It is definitely possible for 7zip or winrar.


I think you have not understood what you have quoted. The point is subtle, so I will try again. In order to zip a floating point value you would first have to save it to a binary (integer) value. To unzip it you would have to reverse this process. In order to recover the original floating point value you must make a number of assumptions about how the hardware and operating system will convert its initial internal representation of a floating point value to an integer, and then how the decompressing machine (which may not be based on the same hardware or even using the same algorithm) will reverse this process.

To put this another way: lets say I give you a double precision PCM stream stored in IEEE754 format. I don't tell you if the machine that created it was operating in double or extended precision mode, if it was intel or amd, or if it was even x86. I then give you the same file, except with the volume doubled (exponent incremented by 1) by the first machine. Can you get a CPU with ARM vfp unit and an Intel Pentium 2 to take the former file, load it into FP registers, double the volume, and then (losslessly) generate the latter? In theory you would think yes, since doubling the volume just increments the exponent without rounding, but that assumes the internal representation of all values and operations is the same. This is a serious question, I really have no idea what would happen on a few different CPUs . . .
Go to the top of the page
+Quote Post
lvqcl
post Jan 4 2011, 21:11
Post #30





Group: Developer
Posts: 3468
Joined: 2-December 07
Member No.: 49183



Both compressing and decompressing can be performed using only integer operations (bit shifts, bitwise AND, etc) or using bitfields:

CODE
struct float32
{
    int mantissa: 23;
    int exponent: 8;
    int sign: 1;
};


This post has been edited by lvqcl: Jan 4 2011, 21:12
Go to the top of the page
+Quote Post
Axon
post Jan 4 2011, 21:49
Post #31





Group: Members (Donating)
Posts: 1985
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



Certainly doing anything meaningful with floating point data requires a "relatively" (heh) inexact measure of equality. That is, there is usually some nonzero amount of relative error to every arithmetic operation you can do with a float. That's part and parcel of using such a format. While it is reasonable to ignore that point when comparing float WAVs, that effectively limits the number of lossless operations one can perform on those WAVs to bit shifts, whereas with fixed point WAVs that would instead be any add followed by an equal-valued subtract (and would explicitly exclude bitshifts extending through the MSB/LSB). IOW, the context here admits two notions of "equality" for float WAVs, not one.

The present discussion, as defined by the OP, is explicitly and solely concerned with maintaining 'absolute' lossless encoding of floating point WAVs, as defined by bit equality (and not by equality to within a relative error or epsilon). That this may not have anything to do with audible sound quality differences (and it likely doesn't!) is entirely besides the point and off topic, for the same reasons why we can have honest discussions about whether or not one sound card or another supports Kernel Streaming or otherwise can send a bit-perfect output signal, even while we might not actually claim that ought to improve audible sound quality.
Go to the top of the page
+Quote Post
indybrett
post Jan 4 2011, 22:22
Post #32





Group: Members (Donating)
Posts: 1354
Joined: 4-March 02
From: Indianapolis, IN
Member No.: 1440



I recall years ago reading (on this board) about non-audio information that WAVPACK would preserve, which FLAC would not. I can't remember if it was some PCM metadata that is only used in a recording studio, or what it was exactly.

It seemed more important to those who were archiving the cd as one file with a cue sheet.

This post has been edited by indybrett: Jan 4 2011, 22:23


--------------------
flac>fb2k>kernel streaming>audiophile 2496>magni>dt990 pro
Go to the top of the page
+Quote Post
lvqcl
post Jan 4 2011, 22:28
Post #33





Group: Developer
Posts: 3468
Joined: 2-December 07
Member No.: 49183



FLAC has --keep-foreign-metadata switch:
QUOTE
If encoding, save WAVE or AIFF non-audio chunks in FLAC metadata. If decoding, restore any saved non-audio chunks from FLAC metadata when writing the decoded file.
Go to the top of the page
+Quote Post
greynol
post Jan 4 2011, 22:31
Post #34





Group: Super Moderator
Posts: 10339
Joined: 1-April 04
From: San Francisco
Member No.: 13167



That's not really on-topic but...

It was with wave files that contain metadata. Generally not (more likely never) applicable to those who embed CUE sheets in lossless images, unless you store a CUE sheet in your wave images (if so tell me how you do this!). flac does announce the error and has since added a switch that can be used to keep this information, IIRC.

This post has been edited by greynol: Jan 4 2011, 22:52


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
indybrett
post Jan 4 2011, 22:31
Post #35





Group: Members (Donating)
Posts: 1354
Joined: 4-March 02
From: Indianapolis, IN
Member No.: 1440



QUOTE (lvqcl @ Jan 4 2011, 16:28) *
FLAC has --keep-foreign-metadata switch:
QUOTE
If encoding, save WAVE or AIFF non-audio chunks in FLAC metadata. If decoding, restore any saved non-audio chunks from FLAC metadata when writing the decoded file.



Yeah, that's it. I think at one time that switch didn't exist, or maybe people just didn't know about it at the time. Either way, that is one example of some (non-audio) data that could be lost if the format doesn't support it, or if the user doesn't know to use a required command line switch.

Edit: verbage

This post has been edited by indybrett: Jan 4 2011, 22:39


--------------------
flac>fb2k>kernel streaming>audiophile 2496>magni>dt990 pro
Go to the top of the page
+Quote Post
greynol
post Jan 4 2011, 22:50
Post #36





Group: Super Moderator
Posts: 10339
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (indybrett @ Jan 4 2011, 13:31) *
or if the user doesn't know to use a required command line switch.

flac does indeed tell you to use the switch when this happens (also keeping on-topic with this off-topic part of the discussion; see the part that Porcus put in italics).

This post has been edited by greynol: Jan 4 2011, 23:06


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Porcus
post Jan 5 2011, 00:54
Post #37





Group: Members
Posts: 1995
Joined: 30-November 06
Member No.: 38207



Even though I originally asked about encoders, I do think that decoding is highly on-topic as well -- if you do get differences when transcoding it does not really matter which of the steps is the culprit.

As for fb2k specifics: Recall that the wiki warns the user against the LA fb2k plugin: http://wiki.hydrogenaudio.org/index.php?title=Lossless_Audio .



BTW, I don't mind spinning this thread off with metadata discussions, but I suppose that it is fairly well known that you cannot safely throw unicode chars into [enter even text file formats here]. (I do think that the 'this couldn't be audible!' argument belongs to the lossies department and not to this thread, though.)


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
googlebot
post Jan 5 2011, 02:00
Post #38





Group: Members
Posts: 698
Joined: 6-March 10
Member No.: 78779



QUOTE (saratoga @ Jan 4 2011, 20:39) *
I think you have not understood what you have quoted. The point is subtle, so I will try again. In order to zip a floating point value you would first have to save it to a binary (integer) value.


That is incorrect. A floating point value can be saved in binary notation without loss. Thus it can be zipped and unzipped without problem. I think you are confusing the floating point numbers with real numbers. Floating point numbers are approximations of real numbers, just in the same way that you can approximate the root of 2 by writing down as many digits as you can fit on a sheet of paper. The representation of floats is well standardized. The only thing differing between machines, as I have written already, is the precision of algorithmic operations and at some special case handling. But this is really not a big deal. The results of two machines after a series of algorithmic operations may differ by some insignificant digits, if you run your operations on the bare bone hardware without additional checks. If consistency is required, most development platforms provide standardized operations which are identical across machines. This is rarely used though, since precision and performance is a higher priority goal than consistency in most scenarios where floating point math is used.

QUOTE (saratoga @ Jan 4 2011, 20:39) *
To unzip it you would have to reverse this process. In order to recover the original floating point value you must make a number of assumptions about how the hardware and operating system will convert its initial internal representation of a floating point value to an integer, and then how the decompressing machine (which may not be based on the same hardware or even using the same algorithm) will reverse this process.


Completely false, sorry.

QUOTE (saratoga @ Jan 4 2011, 20:39) *
To put this another way: lets say I give you a double precision PCM stream stored in IEEE754 format. I don't tell you if the machine that created it was operating in double or extended precision mode, if it was intel or amd, or if it was even x86.


It helps to know whether it was single or double precision. The rest doesn't matter.

This post has been edited by googlebot: Jan 5 2011, 02:08
Go to the top of the page
+Quote Post
Axon
post Jan 5 2011, 02:37
Post #39





Group: Members (Donating)
Posts: 1985
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



For that matter, IIRC, since IEEE 754 demands normalization, which generally makes 7-15 bits of each number fairly coherent across an array, one ought to be able to unambiguously autodetect the endianness of a bytestream of doubles, no matter what numbers are actually encoded. (Unless both the exponent and mantissa truly are randomized, but even then I think you might have outs.)
Go to the top of the page
+Quote Post
2Bdecided
post Jan 5 2011, 11:59
Post #40


ReplayGain developer


Group: Developer
Posts: 5364
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (saratoga @ Jan 4 2011, 18:55) *
The ones and zeros making up a floating point file do not necessarily have a consistent meaning across different CPUs and even operating systems, and the output of identical floating point operations on different systems is not required (or even expected) to be identical. The values represented in each floating point sample are therefore literally approximations of an (unknown) true value that is not recorded.
The sound you can hear is that of goal posts shifting. Let's take it step by step...

QUOTE
The ones and zeros making up a floating point file do not necessarily have a consistent meaning across different CPUs and even operating systems
False. There's no magic. Those ones and zeros mean exactly one number, and it's perfectly well defined what that number is.

QUOTE
the output of identical floating point operations on different systems is not required (or even expected) to be identical.
We're not talking about operations. FWIW I don't expect the result of identical fixed point operations in different audio editors to be identical - they should be dithering, and they won't be using the same PRN or even algorithm.

QUOTE
The values represented in each floating point sample are therefore literally approximations of an (unknown) true value that is not recorded.
How is this different from fixed-point audio samples?

googlebot wrote a more articulate response, but I think the above floors in the argument are even more basic and fundamental.

Cheers,
David.
Go to the top of the page
+Quote Post
bryant
post Jan 5 2011, 21:32
Post #41


WavPack Developer


Group: Developer (Donating)
Posts: 1297
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



I’d like to clarify a couple things about WavPack that were touched upon earlier.

WavPack is 100% lossless with both 32-bit integer and 32-bit floating-point audio files. For floating point this includes handling the special cases of denormals, NaNs, +/- infinities, etc. Basically, in all cases the binary data is preserved regardless of the content. In fact, one of my tests is to change the interpretation of the various data types and make sure everything still works (although, of course, the compression ratio suffers with incorrect interpretation).

As was alluded, floating-point operations (especially divide, I think) are not always guaranteed to produce the same result on all hardware (at least not by default) and this is one reason why WavPack does not use any floating-point math to compress floating-point data. Instead, the float values are converted to the largest 25-bit integers that can be represented without clipping (each block is scanned first) and compressed with the standard integer code, and then a second data stream is created to losslessly store any data that was truncated during the conversion to integers (this second stream goes in the correction file for hybrid lossless).

The -p option mentioned above discards this second stream, making the operation equivalent to 25-bit integers, although since each block is still independently coded with potentially shifted 25-bit “windows” into the floating-point data, it still retains most of the advantage of using floating-point representation. While for all practical purposes this is as good as lossless, it isn’t technically lossless, and is therefore reported as lossy (and is not the default behavior).

Regarding the original question, there actually is a situation in which WavPack will discard information and still claim lossless compression, although it’s a pretty esoteric case. If the wav file header indicates that the audio has fewer significant bits than the container (for example, 20-bit audio in 3 bytes) then only those significant bits are stored, even if the other bits are not zeros. This will show up as a decoding error only if md5 checking is enabled (because that sum is of the raw audio bytes). I don’t believe that this is an issue because all the audio specified in the wav header is losslessly stored, however it was a case that needed special attention for inclusion in WinZip because of the implicit assumptions of a file archiver.
Go to the top of the page
+Quote Post

2 Pages V  < 1 2
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th December 2014 - 18:59