IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Does lossy encoding always produce the same result?, was: "Encoding lossy leads to?"
SamDeRe81
post Mar 5 2011, 03:43
Post #1





Group: Members
Posts: 70
Joined: 24-November 10
Member No.: 85992



I read on the forum that if I run the same file through a lossy encoder twice it will produce the same bit identical files. I want to know if this is true. I thought the algorithm decides what to keep and throw out but it differs each time the encoder is run? What about the container it's stored in isn't that going to be different each time too (aka AAC -> M4A) thereby making them non-identical CRC's? What if it's VBR won't that vary each time the encoder is run? Any input is appreciated it's just a curiosity and want to know.
Go to the top of the page
+Quote Post
Soap
post Mar 5 2011, 05:02
Post #2





Group: Members
Posts: 1015
Joined: 19-November 06
Member No.: 37767



Are you asking (what I answered in the original thread) if LAME A = B the first time and LAME A =B the second time
OR are you asking if LAME A=B the first time and LAME B=B the second time?

For the first is true, but the second is not.


--------------------
Creature of habit.
Go to the top of the page
+Quote Post
NullC
post Mar 5 2011, 07:10
Post #3





Group: Developer
Posts: 200
Joined: 8-July 03
Member No.: 7653



QUOTE (SamDeRe81 @ Mar 4 2011, 19:43) *
I read on the forum that if I run the same file through a lossy encoder twice it will produce the same bit identical files. I want to know if this is true. I thought the algorithm decides what to keep and throw out but it differs each time the encoder is run? What about the container it's stored in isn't that going to be different each time too (aka AAC -> M4A) thereby making them non-identical CRC's? What if it's VBR won't that vary each time the encoder is run? Any input is appreciated it's just a curiosity and want to know.


I don't see why any encoder developer would _intentionally_ make the codec non-deterministic. This would greatly frustrate testing, since it's useful to know that a change which shouldn't have changed the output didn't actually change the output.

Some formats do have some randomness in them. For example, the OGG container has a random serial number (which also effects the CRCs), but the codec output itself should be the same for identical code running on the same kind of system, with identical input. I would not be inclined to trust software that didn't work this way.

Go to the top of the page
+Quote Post
SamDeRe81
post Mar 5 2011, 17:10
Post #4





Group: Members
Posts: 70
Joined: 24-November 10
Member No.: 85992



QUOTE (NullC @ Mar 4 2011, 22:10) *
QUOTE (SamDeRe81 @ Mar 4 2011, 19:43) *
I read on the forum that if I run the same file through a lossy encoder twice it will produce the same bit identical files. I want to know if this is true. I thought the algorithm decides what to keep and throw out but it differs each time the encoder is run? What about the container it's stored in isn't that going to be different each time too (aka AAC -> M4A) thereby making them non-identical CRC's? What if it's VBR won't that vary each time the encoder is run? Any input is appreciated it's just a curiosity and want to know.


I don't see why any encoder developer would _intentionally_ make the codec non-deterministic. This would greatly frustrate testing, since it's useful to know that a change which shouldn't have changed the output didn't actually change the output.

Some formats do have some randomness in them. For example, the OGG container has a random serial number (which also effects the CRCs), but the codec output itself should be the same for identical code running on the same kind of system, with identical input. I would not be inclined to trust software that didn't work this way.


Anyone interested in testing this? Uploading some mp3 and then three or four of us encode it with LAME or something else and check the CRC?
Go to the top of the page
+Quote Post
greynol
post Mar 5 2011, 17:53
Post #5





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



You mean what you should have done before you started this discussion?


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
SonicBooom!
post Mar 5 2011, 18:07
Post #6





Group: Members
Posts: 118
Joined: 16-February 11
Member No.: 88210



QUOTE
Does lossy encoding always produce the same result?, was: "Encoding lossy leads to?"


No, it does not not. Lossless to lossy removes some inaudible sounds (also means "degrading the quality") while lossy to lossy removes it (thus losing more quality) even further.


--------------------
sin(α) = v sound/v object = Mach No.
Go to the top of the page
+Quote Post
db1989
post Mar 5 2011, 18:37
Post #7





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



Read before replying.

The OP is asking whether encoding the same file on two different occasions will produce different results. Yes if one uses different executables, no if one uses the same executables. (Different compiles of the same version can produce minute differences; this is normal and nothing to be concerned about.)

Why would the same executable produce different results every time? Mathematics are deterministic, unless intentionally made otherwise. And why would someone do that?

This post has been edited by dv1989: Mar 5 2011, 18:43
Go to the top of the page
+Quote Post
Rotareneg
post Mar 5 2011, 19:22
Post #8





Group: Members
Posts: 194
Joined: 18-March 05
From: Non-Euclidean
Member No.: 20701



I could see this happening if an encoder and/or decoder used a pseudo-random noise generator (for dithering or whatever) and seeded the generator from the system clock, but I don't know why you'd want to do that. smile.gif

This post has been edited by Rotareneg: Mar 5 2011, 19:28
Go to the top of the page
+Quote Post
SpiderJon
post Mar 5 2011, 19:50
Post #9





Group: Members
Posts: 16
Joined: 8-December 08
Member No.: 64137



QUOTE (dv1989 @ Mar 5 2011, 17:37) *
Why would the same executable produce different results every time? Mathematics are deterministic, unless intentionally made otherwise. And why would someone do that?


Perhaps if the the executable uses a rand() function somewhere, for some reason. (I don't know if any lossy encoder does, but it might well - eg, for dithering.) Even then, the differences shouldn't be detectable in the audible output (at least, not by human listeners).
Go to the top of the page
+Quote Post
SamDeRe81
post Mar 5 2011, 20:13
Post #10





Group: Members
Posts: 70
Joined: 24-November 10
Member No.: 85992



I just tested this with Sound Converter in linux and it did produce the same CRC's, interesting. Thanks

PS I just wanted expert answers greynol that's why I started the thread, but yea thanks everyone be good cheerio!
Go to the top of the page
+Quote Post
pdq
post Mar 5 2011, 22:31
Post #11





Group: Members
Posts: 3404
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



QUOTE (SpiderJon @ Mar 5 2011, 14:50) *
QUOTE (dv1989 @ Mar 5 2011, 17:37) *
Why would the same executable produce different results every time? Mathematics are deterministic, unless intentionally made otherwise. And why would someone do that?


Perhaps if the the executable uses a rand() function somewhere, for some reason. (I don't know if any lossy encoder does, but it might well - eg, for dithering.) Even then, the differences shouldn't be detectable in the audible output (at least, not by human listeners).

Encoders don't dither. You're thinking of decoders.
Go to the top of the page
+Quote Post
greynol
post Mar 5 2011, 22:49
Post #12





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I doubt that decoders generate random outputs either. All the differences I've seen between different decoders have been attributed to rounding at the LSB. I've never seen a decoder that delivers inconsistent output when decoding the same file more than once. Perhaps someone can provide an example to the contrary.


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
NullC
post Mar 5 2011, 23:20
Post #13





Group: Developer
Posts: 200
Joined: 8-July 03
Member No.: 7653



QUOTE (SamDeRe81 @ Mar 5 2011, 09:10) *
QUOTE (NullC @ Mar 4 2011, 22:10) *

I don't see why any encoder developer would _intentionally_ make the codec non-deterministic. This would greatly frustrate testing, since it's useful to know that a change which shouldn't have changed the output didn't actually change the output.

Some formats do have some randomness in them. For example, the OGG container has a random serial number (which also effects the CRCs), but the codec output itself should be the same for identical code running on the same kind of system, with identical input. I would not be inclined to trust software that didn't work this way.


Anyone interested in testing this? Uploading some mp3 and then three or four of us encode it with LAME or something else and check the CRC?


You'll get different results, I qualified "same kind of system" for a reason. Since most lossy codecs use floating point permitted differences in compiler behavior can result in slightly different results from some computations. This is different from run to run determinism.
Go to the top of the page
+Quote Post
lvqcl
post Mar 6 2011, 09:41
Post #14





Group: Developer
Posts: 3382
Joined: 2-December 07
Member No.: 49183



QUOTE (greynol @ Mar 6 2011, 00:49) *
I doubt that decoders generate random outputs either. All the differences I've seen between different decoders have been attributed to rounding at the LSB. I've never seen a decoder that delivers inconsistent output when decoding the same file more than once. Perhaps someone can provide an example to the contrary.

this - http://www.hydrogenaudio.org/forums/index....st&p=746382 ?
Go to the top of the page
+Quote Post
greynol
post Mar 6 2011, 17:45
Post #15





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I'm looking for an example. In that thread I only see the word "if".


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
NwAvGuy
post Mar 7 2011, 15:14
Post #16





Group: Members
Posts: 34
Joined: 2-March 11
Member No.: 88650



I agree with the deterministic software comments--in theory there's no reason why you should get a different audio result (beyond "housekeeping" issues not related to the audio data). Unless there's some randomness intentionally built into the encoder, you should always get the same result on the same PC with the exact same "input" file, with the same encoder software, operating with the same settings.

But if you change any of those variables, all bets are off. Different run time libraries, CPUs, operating systems, etc. could have a subtle impact on the floating point math.

And, it's probably obvious, if you rip a CD twice with lossy encoding you may get a different result if your rips are not 100% bit accurate. If the start of the rip, for example, changes by just one sample, the output files may not be identical. Redbook audio CD's don't have the same file integrity as data CD's. So, to test this, you should use the same lossless file and not CD audio media.


--------------------
Personal Non-Commercial Audio Blog: http://nwavguy.com
Go to the top of the page
+Quote Post
[JAZ]
post Mar 7 2011, 20:32
Post #17





Group: Members
Posts: 1783
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



The "randomness" of floating point is generally overemphasized.

There are just a few differences:

x87 float (which works internally in 80bit and is quite slow)
SSE float (which works internally in 64bit and is fast)

(This matters only when doing several operations over some data in CPU registers, before storing it back to memory)


Then, there is the rounding method:

truncate to zero ( -0.6 = 0 , 0.6 = 0 ) (faster)
round to nearest ( -0.6 = -1, 0.6 = 1 ) (slower)

(You could accumulate a rounding error if working over a variable. That's when the developer has to decide if using double is a better idea)


At last, there is also denormals ( http://en.wikipedia.org/wiki/Denormal_number ), and the action done on them:

keep them. (really slow on Pentium 4s, slow everywhere else)
round them to zero.



Generally, all these options are either defined by the language (IIRC, C++ defines that floats have to use rounding, not truncating, but I can't find a reference to this right now), or by the options set when compiling (A compiler may use SSE floats if compiling for a supported processor and the operation can be done in such way).



There are hardware differences between implementations, but they have to conform to the standard (IEEE-754), so they should not output a different result than the one expected.
Go to the top of the page
+Quote Post
robert
post Mar 13 2011, 21:36
Post #18


LAME developer


Group: Developer
Posts: 788
Joined: 22-September 01
Member No.: 5



The pitfalls of verifying floating-point computations
Go to the top of the page
+Quote Post
saratoga
post Mar 13 2011, 21:48
Post #19





Group: Members
Posts: 4962
Joined: 2-September 02
Member No.: 3264



Depends on the format. WMA for instance does have a random number generator involved in the decoder, however most decoders seed it with the same value every time, so you'll get deterministic output (but of course you're free to not do this if you don't mind having random output).
Go to the top of the page
+Quote Post
hidn
post Mar 13 2011, 22:17
Post #20





Group: Members
Posts: 55
Joined: 17-April 08
Member No.: 52847



you can check it:
http://www.foobar2000.org/
and plug-in
http://www.foobar2000.org/components/view/foo_bitcompare
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th September 2014 - 02:03