IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Proper way to perform a null test
tahaa7
post Apr 15 2014, 01:56
Post #1





Group: Members
Posts: 25
Joined: 7-September 11
From: Europe
Member No.: 93569



What is the best/recommended way to perform a null test between two audio tracks? What program to use? How to exactly align the tracks in time?

Any advice or help is welcome. Thanks.
Go to the top of the page
+Quote Post
saratoga
post Apr 15 2014, 03:42
Post #2





Group: Members
Posts: 4853
Joined: 2-September 02
Member No.: 3264



Almost any audio editor can be used.
Go to the top of the page
+Quote Post
AndyH-ha
post Apr 15 2014, 04:16
Post #3





Group: Members
Posts: 2205
Joined: 31-August 05
Member No.: 24222



One track must be inverted.
The two tracks must be mixed together
or
one copied and pasted over the other.
The only way I know to assure proper alignment is to zoom in to the individual sample level and identify a common starting point in each track. Being off just one sample can make a big difference.
Go to the top of the page
+Quote Post
bandpass
post Apr 15 2014, 08:46
Post #4





Group: Members
Posts: 326
Joined: 3-August 08
From: UK
Member No.: 56644



DiffMaker can do the alignment (and other useful stuff) automatically:

http://www.libinst.com/Audio%20DiffMaker.htm

Go to the top of the page
+Quote Post
AndyH-ha
post Apr 15 2014, 09:48
Post #5





Group: Members
Posts: 2205
Joined: 31-August 05
Member No.: 24222



That sounds like a pretty neat, but frequently impossible, trick.
Go to the top of the page
+Quote Post
Kees de Visser
post Apr 15 2014, 14:41
Post #6





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



The null test is one of my favorite tests. It proves that two audio streams are identical when the difference signal is exactly zero. Perfect nulling is only possible in the digital domain.
If the difference signal isn't zero, all bets are off and all you can do is make an educated guess by listening to or looking at the difference signal.
There can be 3 scenarios:
a) perfect null, the audio streams are identical and should (will!) sound identical. No more listening is necessary.
b) non-perfect null, the audio streams are not identical, but perceptually indistinguishable (until you find a person who can hear a difference).
c) non-perfect null, the audio streams are not identical and perceptually different (ABX-able).

Like the two others said, you need an audio editor (like Reaper), capable of mixing 4 tracks into 2.
-Open file A in tracks 1+2 and file B in tracks 3+4.
-Move one of them until they are perfectly sample-accurately aligned.
-Invert the polarity (phase) of A or B.
-Play all 4 tracks. Listen and watch the stereo output master peak meters. With a bit a luck they won't move. You can move up the master fader to its max to verify there's only silence. SET IT BACK TO ZERO RIGHT AWAY TO PROTECT YOUR EARS AND EQUIPMENT !

If the output meters still move it could be that the files are:
1) not perfectly aligned. Move sample by sample until the output is as low as possible (hopefully -inf.)
2) not the same level. Adjust the level of A or B until the mixer output is as low as possible. Mixing desk faders might not be accurate enough though.
3) simply not identical. In some cases the difference is white noise (dither?), or only certain frequencies, when one of the files has been EQ'd.
4) not sync because the timing difference between A and B is less than one sample. This can e.g. happen with SRC (sample rate conversion). Mac users can try a handy Audio Unit plugin which allows for sub-sample delays (airwindows.com).

If the output isn't zero it can be handy to record the mix to a file for closer examination (like FFT). Make sure that the output format is suitable (24 or 32 bits is fine) and that no processing (like src or dither) is applied.

Hope this helps.
Kees de Visser

This post has been edited by Kees de Visser: Apr 15 2014, 14:47
Go to the top of the page
+Quote Post
drewfx
post Apr 15 2014, 17:21
Post #7





Group: Members
Posts: 74
Joined: 17-October 09
Member No.: 74078



Some other things to add:

1. Be careful how you measure the output, as some programs won't show anything below a certain level. For instance I know Sonar's meters show -∞ below ~-100dBFS even if there is a signal there. In this case it is not a problem for the kind of usage the program is intended for, but is if you're looking for a "perfect null".

2. If you are comparing things like tracks or exports in a DAW, be aware that many things like VST processors or synths contain random processing that will differ on each export. To check for this, you can export the exact same thing twice without changing whatever you're testing for and verify that they null perfectly before proceding.
Go to the top of the page
+Quote Post
DVDdoug
post Apr 15 2014, 19:47
Post #8





Group: Members
Posts: 2535
Joined: 24-August 07
From: Silicon Valley
Member No.: 46454



QUOTE
How to exactly align the tracks in time?
If you have to re-align the tracks, there's a difference and you have already "failed" the null test. wink.gif

The most important thing to remember is that the sound of the difference is NOT the same as the difference in the sound! Two files can sound identical when there are gross differences that show-up in a null test.

There are at least 3 ways to demonstate this:

1. Starting with two bit-identical files, time-delay one copy by a few milliseconds - There is no difference in the sound, but the subtracted difference proves there is a huge difference, and the difference file sounds nothing like a delay. (If you have experience with this stuff, you might recongize it as the side-effect of a delay.)

2. Take two bit-identical files and invert one copy - Again no difference in sound. The difference-file (now "subtracting a negative") sounds identical to the original files, except louder. Again proving a huge difference in the "data" with no difference in sound.

3. Perform a null-test between two completely different-unrelated files. Now the difference-file is simply a mix of the two files that sounds identical to summation of the two files. This could even be two different recordings of the same song, say a live and a studio version, or the same song by two different artists... The null isn't going to tell you anything about the difference between the two versions, you'll just hear both versions mixed.



...I'm NOT saying that null tests are worthless. If you get silence (or a very quiet file) there is no difference (or very little difference) in sound. However, it doesn't work the other way around.

This post has been edited by DVDdoug: Apr 15 2014, 19:58
Go to the top of the page
+Quote Post
Kees de Visser
post Apr 16 2014, 09:34
Post #9





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (DVDdoug @ Apr 15 2014, 19:47) *
If you have to re-align the tracks, there's a difference and you have already "failed" the null test.
Nonono, forget about comparing files. That's too easy with a computer smile.gif
The great advantage of a null test in a DAW (editor) or any other mixer is that you can compare audio streams of different duration, bit depth and even sampling rate (if you accept that upsampling is near-transparent). It's even possible with analog sources, like 2 tape machines, but the null will never be perfect in the analog domain. In this case the null test is used to synchronize two sources, not to proof similarity. I've used this a lot to sync my DAW to an external video machine's audio track.
Go to the top of the page
+Quote Post
Kees de Visser
post Apr 17 2014, 13:55
Post #10





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



I've done a little null testing with a DXD (24 bit 352.8 kHz) source (Haydn from the 2L.no site) and a self made 44.1 version. It can be shown that the difference is at a very low level, about 15 dB below 24-bit dither. So if hi-res formats have any benefits, it has to be the extended bandwidth above 20kHz, because below 20 kHz there is no difference to speak of (at least not in the digital domain).


Go to the top of the page
+Quote Post
AndyH-ha
post Apr 17 2014, 23:30
Post #11





Group: Members
Posts: 2205
Joined: 31-August 05
Member No.: 24222



QUOTE
If you have to re-align the tracks, there's a difference and you have already "failed" the null test.
Many times a file will contain some samples at the beginning or the end that were not in the original. This is probably the norm when extracting from CD. Perhaps it has to with offsets. Regardless, these are always either zero values or very low level noise, but the principal would be the same if it were a burst of very loud static or a curse from Satan. Or, maybe a few bits will be lost instead of added.

Even so, the audio data of interest may be, and generally will be, barring damaged media, bit for bit identical with the original source to which one wants to compare. However, if the correct alignment is not achieved before adding the two files, the results will be very different than the string of zero values that mean the two are identical.

If you want to argue that the two were not identical because of the added or missing bits at the end point(s), you are free to live in that universe, but in most cases you are not obtaining very useful information.

If the answer to 'how to align' is still open, this is sometimes difficult. If the goal is to find out if one can extract accurately, one can create test data by
pulling up one sample (say 20dB) near the beginning of each track (or pulling down, if the track opening is very loud), creating a very easy to identify sample on which to align
writing these modified tracks to CD-R,
extract them from the CD-R,
then run the test on each extracted against source track.

If one has tracks where some particular beginning sample is too difficult to identify, one can make a best guess and do the test. If the result is not nulled, one can move one of the tracks by one bit and try again. This can be very tedious if one is using too vague a starting point, but it can work if one is persistent.

Best in this case is to go to some identifiable samples to find a start point. It doesn't matter if this means ignoring the first second or two, or whatever. If one get null values by starting at this identified place, one can then work backwards to find out when the tracks begin to differ. A good audio editor will show the sample count. Thus one can easily step forward or backwards by 151,329 samples, or any other particular number, allowing a more reasonable search procedure.

Go to the top of the page
+Quote Post
Kees de Visser
post Apr 18 2014, 08:02
Post #12





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (AndyH-ha @ Apr 17 2014, 23:30) *
This can be very tedious if one is using too vague a starting point, but it can work if one is persistent.
Another option is to insert a sample-accurate delay plugin into one of the stereo pairs, play both pairs and adjust the delay on the fly until both streams are perfectly sync. With a bit of training this can be done within a minute. When sync is far off you hear double sources, like an echo. When sync is close, the mix becomes colored (comb filtering) and when sync is perfect (within one sample) the mix output will be silence or very low level, depending on how identical the two sources are.
Go to the top of the page
+Quote Post
splice
post Apr 18 2014, 10:13
Post #13





Group: Members
Posts: 119
Joined: 23-July 03
Member No.: 7935



I used a similar trick back in analogue tape days. I wired up a short headphone extension lead with the ground connection lifted. I'd play a test tape, or the tape which I needed to align the deck to, and adjust the azimuth for minimum output. The advantage was that it worked with normal program material and was quicker than and as accurate as peaking for maximum HF.


--------------------
Regards,
Don Hills
Go to the top of the page
+Quote Post
krabapple
post Apr 18 2014, 16:20
Post #14





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



Adobe Audition has fairly easy to use 'invert /mixpaste' option that's perfect for null tests. (You still have to do alignment yourself though, if it needs doing).



Adobe was *giving away* Audition awhile back on its site...don't know if that's still true.

This post has been edited by krabapple: Apr 18 2014, 16:21
Go to the top of the page
+Quote Post
2Bdecided
post Apr 22 2014, 11:03
Post #15


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (krabapple @ Apr 18 2014, 16:20) *
Adobe Audition has fairly easy to use 'invert /mixpaste' option that's perfect for null tests. (You still have to do alignment yourself though, if it needs doing).


Be aware that the above works, BUT an apparent alternative does not: You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)

Hence if you compare bit-identical clipped audio tracks by inverting one separately first, then when you add it to the non-inverted track, it will not perfectly null, but will leave differences with an amplitude of one least significant bit.

Cheers,
David.
Go to the top of the page
+Quote Post
krabapple
post Apr 22 2014, 21:16
Post #16





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



The function is actually called just 'mixpaste', but it includes the option to check an 'invert' box.
Go to the top of the page
+Quote Post
db1989
post Apr 22 2014, 21:25
Post #17





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (2Bdecided @ Apr 22 2014, 11:03) *
You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)
Further to the discussion of this held here previously, I’m wondering if there were ever any rough stats on how many recordings ever actually use that lowest value? I completely understand why 2’s-complement is used in computing, but it seems to me the best idea is to eschew use of the most negative number, limiting that side to the same absolute maximum as the positive. Parameter scales that are asymmetrical for this reason really offend my desire for neatness! laugh.gif And I guess now I’m off to search for philosophical debates about this number wink.gif

This post has been edited by db1989: Apr 22 2014, 21:30
Go to the top of the page
+Quote Post
Kees de Visser
post Apr 22 2014, 22:48
Post #18





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (2Bdecided @ Apr 22 2014, 11:03) *
You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)

Interesting, I've never encountered that problem, so I did a quick test:
-Create a full-scale white noise 16/44.1 aiff file.
-Increase level 3 dB to create plenty of overloads and save file (still 16/44.1)
-Create a phase inverted copy (16/44.1)
-Mix both files (1:1) and save as new file (16/44.1)

The difference file contains no signal.

My practice doesn't seem to agree with your theory. Any idea why ?
Go to the top of the page
+Quote Post
pdq
post Apr 23 2014, 00:00
Post #19





Group: Members
Posts: 3372
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



You start with 65536 possible values, and you end with 65536 possible values, so there is a 1:1 mapping that will work.

Namely, if you perform a 1's complement then +32767 becomes -32768, -32768 becomes +32767, 0 becomes -1 and -1 becomes 0, etc.

I don't know if this is how the problem is generally avoided, but it certainly would work.
Go to the top of the page
+Quote Post
xnor
post Apr 23 2014, 01:29
Post #20





Group: Developer
Posts: 490
Joined: 29-April 11
From: Austria
Member No.: 90198



No, the problem is avoided by doing these operations at higher bit depths. Audition does this automatically.

0 = 0, (-1)*1 = -1, (-1)*32767 = -32767 and (-1)*-32768 = clipped to 32767.

In Audition you need to save both normal and inverted as 16-bit files and reopen them, then mix-paste and you'll see it will only null to -1 on the negative side.
(to make sure the inverted file isn't dithered on save create it as 24+ bit, paste, invert, convert it to 16 bit without dither, then save)

This post has been edited by xnor: Apr 23 2014, 01:34
Go to the top of the page
+Quote Post
[JAZ]
post Apr 23 2014, 09:40
Post #21





Group: Members
Posts: 1751
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



So, the last few posts are with, or against 2BDecided's post?

From the way they are redacted, it is not clear. From the implications, It seems obvious that he is correct:

Case A) Program does 2's complement (binary inversion). 32767 becomes -32768, 0 becomes -1, 1 becomes 0. 0 + -1 = -1 , not 0, 32767 + -32768 becomes -1.
Case C) Program does 2's complement and clips: 32767 becomes -32767, 32766 becomes -32767, 0 becomes -1, 1 becomes 0. 0 + -1 = -1 32767 + -32767 = 0.
Case C) Program invert sign (decimal inversion) and clips. 32767 becomes -32767 becomes -32767, 32766 becomes -32766, 0 becomes 0, 1 becomes 1, -32768 becomes -32767. 0 + 0 = 0, 32767 + -32767 = 0. -32768 + 32767 = -1

In all the cases, one cannot be accurate up to the LSB (least significant bit). So, obviously, when converting 16bit to 24 bit and using case C, it works. In any of the other cases, it does not.

I repeat: invert paste, with integer, does not work down to the LSB.

This post has been edited by [JAZ]: Apr 23 2014, 09:46
Go to the top of the page
+Quote Post
2Bdecided
post Apr 23 2014, 09:58
Post #22


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (krabapple @ Apr 22 2014, 21:16) *
The function is actually called just 'mixpaste', but it includes the option to check an 'invert' box.
Yes, and that's the one that works.

The separate invert function does not. In CEP 1.2a anyway. I'm a Luddite. wink.gif


QUOTE (pdq @ Apr 23 2014, 00:00) *
You start with 65536 possible values, and you end with 65536 possible values, so there is a 1:1 mapping that will work.

Namely, if you perform a 1's complement then +32767 becomes -32768, -32768 becomes +32767, 0 becomes -1 and -1 becomes 0, etc.

I don't know if this is how the problem is generally avoided, but it certainly would work.
It would work for this specific use case, but more generally it would be strange to have an invert function that introduced a DC offset. I guess alternatively it's strange that the invert function potentially introduces (insignificant) clipping.

Aside from mathematical analysis, I can't imagine that anyone ever cared about these limitations.


QUOTE (db1989 @ Apr 22 2014, 21:25) *
QUOTE (2Bdecided @ Apr 22 2014, 11:03) *
You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)
Further to the discussion of this held here previously, Iím wondering if there were ever any rough stats on how many recordings ever actually use that lowest value? I completely understand why 2ís-complement is used in computing, but it seems to me the best idea is to eschew use of the most negative number, limiting that side to the same absolute maximum as the positive. Parameter scales that are asymmetrical for this reason really offend my desire for neatness! laugh.gif
Me too. But in the context of the invention of the CD (i.e. pre DAW, pre audio in PCs etc), I guess none of this mattered. You would never have an absolutely zero DC offset in a practical A>D, and you would never be able to ensure that you just hit 32767 or 32768 - most early CDs had headroom between the maximum signal level and 0dB FS.

Cheers,
David.
Go to the top of the page
+Quote Post
pdq
post Apr 23 2014, 14:15
Post #23





Group: Members
Posts: 3372
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



QUOTE (2Bdecided @ Apr 23 2014, 04:58) *
QUOTE (krabapple @ Apr 22 2014, 21:16) *
The function is actually called just 'mixpaste', but it includes the option to check an 'invert' box.
Yes, and that's the one that works.

The separate invert function does not. In CEP 1.2a anyway. I'm a Luddite. wink.gif

Clearly then the difference is that it is OK to subtract -32768, but not to negate it first and then add it.
Go to the top of the page
+Quote Post
xnor
post Apr 23 2014, 14:48
Post #24





Group: Developer
Posts: 490
Joined: 29-April 11
From: Austria
Member No.: 90198



[JAZ], the proper way to negate a 2's complement number is by flipping all bits and adding 1.
Since 32767+1 = -32768, there is no way to invert -32768.

I don't think programs internally operate with 16 bit integers, but more like 32 or even 64 bit ones (possibly even floating point), so you don't need any special inversion code.

I see no problem with invert, mix-paste. If your program fails this test you can always convert the files to a higher bit depth.

This post has been edited by xnor: Apr 23 2014, 14:49
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th July 2014 - 12:51