Welcome Guest ( Log In | Register )

Pathological example of a intersample peak, 11dB, discussion.
post Jan 10 2013, 04:36
Post #1

Group: Members
Posts: 77
Joined: 13-January 09
Member No.: 65515

Pathological example of a intersample peak that was artificially created:

~0dB peak, ~20dBFS RMS (squarewave), +10.87dB intersample peak, 44.1KHz, 32bit float.


Please keep any discussion of the test sample in this tread, rather than where it's simply "stored".

The problem:
If oversampled the true peak is reveal to be almost +11dB.
A DAC would need 11dB headroom (or alternatively ~12dB which equals 2 bits) to handle this wav correctly.

The solution?:
A "quick fix" for a 24bit (or float) audio chain, would be to reduce the volume by 12dB somewhere.
Volume loss can be later compensated by simply increasing the analog volume (the user turning the knob a little higher).

*** The rest is somewhat opinionated. ***

As such the "bottom 3 bits" of audio could be considered waste-able, 2 bits to handle pathological intersample peaks, + 1 bit due to quantization/noisefloor/dither.
A "24bit" DAC would have no issues, 21bits to use is a lot. Likewise a "20bit" DAC would still have 17bits to use.
Ideally the 11 (or 12) dB volume reduction would be done by the DAC just before the reconstruction stage.

For a 12dB headroom DAC one would need to crank up the playback volume, so such a DAC would sound more quiet than most other DACs.
Noisefloor of the amplifier and other parts of the equipment/audio chain is also an issue.
But even "cheap" gear has around -80dBFS to -100dBFS noisefloor.
Also considering that a normal living room can easily have a +50dB noisefloor, so loosing out on the 12dB or so of the quietest audio is not an issue.

So if taking CD audio as an example, a 12dB adjustment would cause the content in the -96dBFS to -84dBFS range to be lost.
The loss can be avoided by simply passing the 16bit audio as 24bit or 32bit float instead.
Under Windows Vista and Windows 7 and Windows 8 all audio is changed to 32bit so this is a non-issue.

How to avoid intersample peaks on gear without the needed headroom?:
On Windows you can simply make sure that you never raise the volume (in Windows) above -12dBFS (~45% volume),
and instead use the analog volume knob (if there is one on your system or gear) instead.

11dB really?
Yep! Then again this is a pathological example.
"Normally" the intersample peak is within 1dB of the digital peak, and in some rare cases up to 2 to 3dB higher.
If you make/master music, then the final mix/pressing master/encoding/exporting should have 2 or 3dB headroom.
So as long as no peaks go above 3dB you should be pretty darn safe from causing any clicks or distortion for the end user.

The example here is a pure spike, and humans tend not to like to listen to pops, clicks, static, test tones, or similar.
So encountering anything like this "in the wild" is very rare.

Is it really that bad?:
Please remember that intersample peaks do not damage equipment, at least I've never heard or read about such happening, and the CD was invented like ages ago.
So if this was a practical issue we'd have heard about it along time ago as equipment got fried etc. And we'd have had a solution years ago as well.

The only thing it does is damage the audio quality, that is if you actually can hear/notice it at all. You are more likely to hear crackling/distortion from overly compressed music.
And ironically it is that type of overly compressed music that has the most intersample peaks that go above 0dBFS.
Solution? Stop compressing the hell out of music. Use 20dB or more headroom and intersample peaks will most likely never be an issue.

Go to the top of the page
+Quote Post
Start new topic
post Jan 14 2013, 00:51
Post #2

Group: Members
Posts: 77
Joined: 13-January 09
Member No.: 65515

bandpass, 2bdecided, saratoga and John Siau will hopefully find this interesting.

4th post here http://www.hydrogenaudio.org/forums/index....st&p=820447
has a .CSV with the raw data from a scan.

Here is the highlights:
5824 tracks.
Peak -1.18 dBFS (Min -28.03, Max 0.00)
ISP -0.45 dBFS (Min -28.02, Max 9.54)
ISP/Peak Delta -1.63 dB (Min -56.05, Max 9.54)
RMS -16.89 dBFS (Min -46.18, Max -6.34)

27.59% ISP <-1 dBFS
72.41% ISP >-1 dBFS

44.33% ISP -1 to 0 dBFS
19.95% ISP 0 to 1 dBFS
3.71% ISP 1 to 2 dBFS
0.79% ISP 2 to 3 dBFS
0.21% ISP 3 to 4 dBFS
0.34% ISP 4 to 5 dBFS
0.31% ISP 5 to 6 dBFS
1.56% ISP 6 to 7 dBFS
1.18% ISP 7 to 8 dBFS
0.00% ISP 8 to 9 dBFS
0.02% ISP 9> dBFS

The tracks are of all genres from "typical" to really weird. WAV, FLAC, Mp3, AAC/MP4, Ogg, 44.1KHz & 48KHz, 16 and 24/32bit. Soundtracks, pop, techno, anime, classic, game, computer music, my own composed music, standup, stereo and mono tracks, multi-channel tracks, spanning multiple decades. It took 4+ hours using 6 cores each at 100% (would have taken 24hrs if only one core had been used).

I am really surprised that only around 30% of the tracks is actually below the -1.0 dBFS for "true peaks" that EBU R128 wants.
I am even more surprised to see around 20% in the 0 to +1 dBFS range as that is (in my opinion) a rater high percentage of ISP "overs" to pass to a DAC or a audio chain.
The 3.71% in the +1 to +2 range and the 0.79% in the +2 to +3 range is very worrying, but a DAC like that mentioned by John Siau should handle this fine.

Where it really gets creepy is the >3dBFS ISPs, 3.62% total in the +3 to >+9 dBFS range. Wit the highest ISP at 9.54.
Those are probably really messed up tracks. Stupidly enough I did not also log the filepaths. (I just created the ids) So I can't easily find out what tracks are this bad so easily.
I may rescan everything in the future (maybe with better tools, R128 scan and log tool would be ideal for this stuff) and if the tracks where commercial tracks I'll post which ones (and if it turns out to be my music I'll provide samplers for testing).

Note! I used a intermediary tool that called sox, the tool itself was called/used by foobar2000 as if it was a encoder/converter, 32bit wav audio data was passed to sox, so it's unlikely the very high ISPs detected are corrupted file data, or if it is then foobar2000 treated it as part of the audio, in which case it matches real world situations (playing of corrupted data as audio).

Hopefully you guys find the .csv interesting, 5824 tracks is a rather large sample of data and thus hopefully useful.
Maybe if others could scan and make similar .csv and we can start gathering large amounts of data for numbers crunching (a foobar plugin or a standalone scan tool would make this easier for folks, *hint hint* !).
Go to the top of the page
+Quote Post
post Jan 14 2013, 13:00
Post #3

ReplayGain developer

Group: Developer
Posts: 5663
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409

QUOTE (Rescator @ Jan 13 2013, 23:51) *
bandpass, 2bdecided, saratoga and John Siau will hopefully find this interesting.

4th post here http://www.hydrogenaudio.org/forums/index....st&p=820447
has a .CSV with the raw data from a scan.

Thanks for this.

Are you seeing what I'm seeing? i.e. do an X-Y plot of column B against column D.

I see two series emerge - one effectively a Y=X line, but another a Y=X+8dB line. What's that? Am I doing something wrong, or not thinking, or is there an error in some of the data (maybe in the decoding of one format)?

From the X=Y line, and where the points start to depart from it (i.e. where inter-sample peaks start to become significantly higher than on-sample peak values), it seems that inter-sample peaks are (mostly) only a problem for tracks in your collection with on-sample peaks above -1dB. i.e. it's only tracks that were (nearly) clipped which generate intersample overs (with a small handful of exceptions).


This post has been edited by 2Bdecided: Jan 14 2013, 13:01
Go to the top of the page
+Quote Post
post Jan 14 2013, 16:49
Post #4

Group: Members
Posts: 77
Joined: 13-January 09
Member No.: 65515

QUOTE (2Bdecided @ Jan 14 2013, 13:00) *
Are you seeing what I'm seeing? i.e. do an X-Y plot of column B against column D.

No! I Do not have MatLAB or whatever. (can't afford to pay thousands).
A quick search for tools on the net turned up nothing for windows. (and I'm not keen to start compiling gnuplot, or doing CSV to JSON conversion to use Googles API etc.)
So if you could point me to a tool for windows, or show an image that would be helpful smile.gif

As to the anomalies you seem to point out. Sox just processed whatever foobar gave it.
It is possible that either foobar passed bad audio/wav data to sox (my code did not alter any data at all, clean passtrough), or that Sox choked on some bitdepth/frequency/channel combo.
I probably won't rerun this test anytime soon (6 cores at 100% for 4 hours is a little on the heavy side) until a proper toolset is available. Maybe the R128 gain tool could be modified to append csv data lines to a file.

Scanning for "true peaks" the way that EBU R128 recommend is of particular interest obviously.
I could make a tool myself, but I have not found any example codes on upsampling and peak scanning, and I don't really read "math" equations.
The tiny tool I made and hooking it up between foobar and sox and the scanning took me a day. If I'm to spend any more time then it's better to do it right, read up and code a program program, and then it's suddenly about a week (or more) of work involved, (modifying an existing tool might be easier for existing maintainers, for me it would probably take me a week or so if unlucky, to learn/read the existing code enough to modify it).

I'd love to see hundreds of people on HA scan their collections, provide the csv and then someone can do some serious number crunching and present the results.
Depending on how many would do the scan and the size of peoples collection/test size the resulting data could number anywhere from a hundred thousand to a million tracks, which is defiantly statistically significant.

This post has been edited by Rescator: Jan 14 2013, 17:05
Go to the top of the page
+Quote Post
post Feb 4 2013, 12:45
Post #5

Group: Members
Posts: 403
Joined: 23-January 05
From: The Netherlands
Member No.: 19254

QUOTE (Rescator @ Jan 14 2013, 16:49) *
QUOTE (2Bdecided @ Jan 14 2013, 13:00) *
Are you seeing what I'm seeing? i.e. do an X-Y plot of column B against column D.

No! I Do not have MatLAB or whatever. (can't afford to pay thousands).

Off-topic, but take a look at GNU Octave. It's a free Matlab-clone, with the same syntax (so your old .m scripts still work).
Go to the top of the page
+Quote Post

Posts in this topic
- Rescator   Pathological example of a intersample peak, 11dB, discussion.   Jan 10 2013, 04:36
- - Rescator   http://forums.digitalspy.co.uk/showthread....25#po...   Jan 10 2013, 07:51
- - Kees de Visser   At a first glance it seems that this test signal i...   Jan 10 2013, 07:57
- - lvqcl   http://www.hydrogenaudio.org/forums/index....st...   Jan 10 2013, 15:52
- - 2Bdecided   While we're quoting past debates, I remember o...   Jan 10 2013, 16:06
- - John_Siau   When designing Benchmark's new DAC2 HGC D/A co...   Jan 10 2013, 16:10
|- - saratoga   QUOTE (John_Siau @ Jan 10 2013, 10:10) I ...   Jan 10 2013, 17:07
|- - Rescator   QUOTE (John_Siau @ Jan 10 2013, 16:10) A ...   Jan 11 2013, 11:27
|- - Banned   QUOTE (Rescator @ Jan 11 2013, 11:27) it ...   Jan 12 2013, 12:24
|- - Rescator   QUOTE (Banned @ Jan 12 2013, 12:24) QUOTE...   Jan 12 2013, 13:14
|- - Banned   QUOTE (Rescator @ Jan 12 2013, 13:14) Nic...   Jan 12 2013, 15:54
|- - Rescator   QUOTE (Banned @ Jan 12 2013, 15:54) Examp...   Jan 12 2013, 18:14
|- - Banned   QUOTE (Rescator @ Jan 12 2013, 18:14) QUO...   Jan 12 2013, 20:52
|- - Rescator   QUOTE (Banned @ Jan 12 2013, 20:52) QUOTE...   Jan 12 2013, 22:53
|- - 2Bdecided   QUOTE (Rescator @ Jan 12 2013, 21:53) To ...   Jan 14 2013, 12:47
- - bennetng   I've seen +4.8dB intersample peak in a song, b...   Jan 10 2013, 18:19
- - 2Bdecided   I don't see how on-sample values above 0dB FS ...   Jan 10 2013, 19:56
- - bennetng   I just generated another synthetic example wavefor...   Jan 10 2013, 20:26
|- - Rescator   QUOTE (bennetng @ Jan 10 2013, 20:26) I j...   Jan 12 2013, 13:22
|- - bennetng   QUOTE (Rescator @ Jan 12 2013, 20:22) QUO...   Jan 12 2013, 16:36
- - bandpass   $ sox InterSamplePeak.wav -n gain -11.9 stats...   Jan 11 2013, 13:13
- - bandpass   Here's how to do the EBU upsampling/filtering ...   Jan 12 2013, 23:17
|- - Rescator   QUOTE (bandpass @ Jan 12 2013, 23:17) Her...   Jan 12 2013, 23:22
- - Rescator   bandpass, 2bdecided, saratoga and John Siau will h...   Jan 14 2013, 00:51
|- - 2Bdecided   QUOTE (Rescator @ Jan 13 2013, 23:51) ban...   Jan 14 2013, 13:00
||- - Rescator   QUOTE (2Bdecided @ Jan 14 2013, 13:00) Ar...   Jan 14 2013, 16:49
||- - bug80   QUOTE (Rescator @ Jan 14 2013, 16:49) QUO...   Feb 4 2013, 12:45
|- - John_Siau   QUOTE (Rescator @ Jan 13 2013, 18:51) ban...   Jan 14 2013, 18:22
- - bandpass   Using LibreOffice (free and available on Windows):   Jan 14 2013, 17:06
- - 2Bdecided   Yep, that's exactly it. (I just used Excel). ...   Jan 14 2013, 17:20
|- - Rescator   (@bandpass Darn you, stop teaching me new tricks. ...   Jan 14 2013, 20:29
|- - bandpass   QUOTE (Rescator @ Jan 14 2013, 19:29) if ...   Jan 15 2013, 09:57
- - John_Siau   The "+11 dB" test signal (that started t...   Jan 14 2013, 18:40
|- - Rescator   QUOTE (John_Siau @ Jan 14 2013, 18:40) Th...   Jan 14 2013, 19:37
- - Wombat   After reading about that topic a bit i found that ...   Jan 14 2013, 22:31
|- - Alexey Lukin   QUOTE (Wombat @ Jan 14 2013, 17:31) After...   Feb 4 2013, 19:49
- - knutinh   Since inter-sample overshoot is a problem for the ...   Feb 4 2013, 09:20
|- - 2Bdecided   QUOTE (knutinh @ Feb 4 2013, 08:20) Since...   Feb 4 2013, 13:08
|- - knutinh   QUOTE (2Bdecided @ Feb 4 2013, 13:08) QUO...   Mar 4 2013, 13:58
|- - John_Siau   QUOTE (knutinh @ Mar 4 2013, 07:58) I vag...   Mar 4 2013, 17:51
|- - 2Bdecided   QUOTE (knutinh @ Mar 4 2013, 12:58) I vag...   Mar 4 2013, 18:21
- - John_Siau   QUOTE (2Bdecided @ Feb 4 2013, 07:08) QUO...   Feb 4 2013, 18:08
- - Rescator   I'm performing some topic necromancy to let th...   Yesterday, 11:59
- - Rescator   I uploaded a 5 sec clip from Savant's Ism here...   Yesterday, 12:28

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:


RSS Lo-Fi Version Time is now: 1st December 2015 - 01:56