IPB

Welcome Guest ( Log In | Register )

8 Pages V  « < 2 3 4 5 6 > »   
Reply to this topicStart new topic
libebur128 - (yet another) EBU R 128 implementation
benski
post Feb 6 2011, 20:51
Post #76


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



Raiden.

For gated loudness of a segment, the EBU R128 spec requires a 50% window overlap. Maybe I'm missing something obvious - but I'm not seeing any indication of overlap in the code. Am I wrong?

This post has been edited by benski: Feb 6 2011, 20:53
Go to the top of the page
+Quote Post
Raiden
post Feb 6 2011, 22:16
Post #77





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



QUOTE (benski @ Feb 6 2011, 20:51) *
For gated loudness of a segment, the EBU R128 spec requires a 50% window overlap. Maybe I'm missing something obvious - but I'm not seeing any indication of overlap in the code. Am I wrong?

The code badly needs some comments, I know. smile.gif
The lib uses a circular buffer of 400ms (or 3s if mode S is enabled). All 200ms a new gating block is calculated, except at the beginning, where 400ms are needed. This behaviour is controlled by the variable "needed_frames", which is initialized to samplerate*2/5 (line 168) and set to samplerate/5 for all following blocks (line 407).

I did a test with some short samples, logging the number of calculated blocks:
CODE
$ sox -n sine.wav synth 0.3999 sine
$ ./r128-sndfile sine.wav
global loudness: -inf LUFS

$ sox -n sine.wav synth 0.4 sine
$ ./r128-sndfile sine.wav
1
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.5999 sine
$ ./r128-sndfile sine.wav
1
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.6 sine
$ ./r128-sndfile sine.wav
1
2
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.7999 sine
$ ./r128-sndfile sine.wav
1
2
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.8 sine
$ ./r128-sndfile sine.wav
1
2
3
global loudness: -3.69 LUFS


This post has been edited by Raiden: Feb 6 2011, 22:18
Go to the top of the page
+Quote Post
benski
post Feb 6 2011, 22:36
Post #78


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



And in the example of a 600ms input, it should be calculating the loudness for 0-400 ms and then 200-600 ms. Because in the code it looks like it might be doing 0-400ms and then 400-600ms. But again I might just be following along wrong - the #define macros made it hard to debug and follow along smile.gif
Go to the top of the page
+Quote Post
kode54
post Feb 7 2011, 00:44
Post #79





Group: Admin
Posts: 4580
Joined: 15-December 02
Member No.: 4082



Ah, right, then. Well, then it would be helpful to add modes for retrieving momentary and short-term loudness, gated. Or maybe I'll just stick to using a fork that does what I need. Because using BS.1770 for normalization instead of track or album analysis is probably beyond its design scope anyway.
Go to the top of the page
+Quote Post
Raiden
post Feb 7 2011, 02:06
Post #80





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



QUOTE (googlebot @ Feb 6 2011, 11:46) *
PS An integrated mode live meter basically doesn't do anything different than a track based I-mode calculation. It has start, stop, and reset buttons to delimit the range.

Thanks, I was just curious how that works. It could be interesting to plot short-term and integrated loudness and see how they influence each other.
QUOTE (googlebot @ Feb 6 2011, 11:46) *
PPS You don't have to redo the whole calculation each time. The absolute gating threshold does not change and must only be compared once per block. The relative gating threshold must be updated and re-applied after every new block. You may hit a drawback of using plain ANSI C, though. C++ would allow comfortable storing of block energies of virtually infinite number in dynamically growing memory structures like vectors or lists. It's not trivial to match the latter in performance/maintainability by hand coding in plain C. But when you've come so far, I'm sure you are going to find a solution.

I currently do something like this already. All block energies above the absolute threshold are stored in a singly linked list. When ebur128_loudness_global is called, this list is traversed twice, first for the relative threshold and a second time for the gated loudness. And yes, I miss std::list. smile.gif
Go to the top of the page
+Quote Post
Raiden
post Feb 7 2011, 02:21
Post #81





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



QUOTE (benski @ Feb 6 2011, 22:36) *
And in the example of a 600ms input, it should be calculating the loudness for 0-400 ms and then 200-600 ms. Because in the code it looks like it might be doing 0-400ms and then 400-600ms. But again I might just be following along wrong - the #define macros made it hard to debug and follow along smile.gif

ebur128_calc_gating_block in line 390 is always called with samplerate*2/5 (400ms) as a parameter. So it will always analyse the last 400ms for each new block. A bit difficult are lines 256-275: I must check if the last 400ms "wrap around" the circular buffer, and handle that case separately.

Another test:
CODE
$ sox -n sine.wav synth 0.2 sine
$ sox -n noise.wav synth 0.2 noise
$ sox sine.wav noise.wav sine.wav noise.wav sine.wav noise.wav sine.wav noise.wav test.wav
$ ./r128-sndfile test.wav
0.654417
0.654412
0.654412
0.654412
0.654412
0.654412
0.654412
global loudness: -2.53 LUFS

The first block differs from the others. This is probably because the filter states must get filled.
Go to the top of the page
+Quote Post
Raiden
post Feb 7 2011, 02:40
Post #82





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



QUOTE (kode54 @ Feb 7 2011, 00:44) *
Ah, right, then. Well, then it would be helpful to add modes for retrieving momentary and short-term loudness, gated. Or maybe I'll just stick to using a fork that does what I need. Because using BS.1770 for normalization instead of track or album analysis is probably beyond its design scope anyway.

How does gated short-term or momentary loudness work? Isn't gating only used for long audio segments where very quiet or silent parts could distort the result?
Go to the top of the page
+Quote Post
romor
post Feb 7 2011, 03:08
Post #83





Group: Members
Posts: 668
Joined: 16-January 09
Member No.: 65630



QUOTE (Raiden @ Feb 7 2011, 02:06) *
It could be interesting to plot short-term and integrated loudness and see how they influence each other.

It I understand you correctly something like that was already posted in that other R128 thread but without short-term history view: http://www.nugenaudio.com/images/VisLM-H.png
It can also output CSV with all kinds of data, but it's commercial

[edit] actually very commercial, I wasn't aware of it's price until now

This post has been edited by romor: Feb 7 2011, 03:39


--------------------
scripts: http://goo.gl/M1qVLQ
Go to the top of the page
+Quote Post
benski
post Feb 7 2011, 05:22
Post #84


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



QUOTE (Raiden @ Feb 6 2011, 20:21) *
QUOTE (benski @ Feb 6 2011, 22:36) *
And in the example of a 600ms input, it should be calculating the loudness for 0-400 ms and then 200-600 ms. Because in the code it looks like it might be doing 0-400ms and then 400-600ms. But again I might just be following along wrong - the #define macros made it hard to debug and follow along smile.gif

ebur128_calc_gating_block in line 390 is always called with samplerate*2/5 (400ms) as a parameter. So it will always analyse the last 400ms for each new block. A bit difficult are lines 256-275: I must check if the last 400ms "wrap around" the circular buffer, and handle that case separately.
The first block differs from the others. This is probably because the filter states must get filled.


I looked at the code again and it's definitely not doing any sort of window overlap. It's treating each 200ms block of audio separately, except for the first block which is done as 400ms. We should get this fixed. Unfortunately, the IIR filter code is writing the filtered data back into the audio_data buffer. In order to do overlapped windows, you'd need to keep the audio intact as each sample will be used twice. One simple solution would be to do the sum-of-squares inside the filter function, so that you don't need to store the filtered audio.

Just to be clear, this is essentially what needs to be done for overlap. For the sake of clarity, let's use a 1000Hz sampling rate.
CODE
ebur128_filter_double(st, &audio_data[0], 400);
ebur128_filter_double(st, &audio_data[200], 400);
ebur128_filter_double(st, &audio_data[400], 400);
ebur128_filter_double(st, &audio_data[600], 400);
/* etc. */

Go to the top of the page
+Quote Post
Raiden
post Feb 7 2011, 06:36
Post #85





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



It is doing window overlap. Otherwise, the block energies from the test in post 81 would be different, and not all 0.654412.
Go to the top of the page
+Quote Post
kode54
post Feb 7 2011, 15:06
Post #86





Group: Admin
Posts: 4580
Joined: 15-December 02
Member No.: 4082



The library doesn't like it very much if the sample rate isn't evenly divisible by 2.5 or 5. Everything works fine if I change every *2/5 to a /5*2.
Go to the top of the page
+Quote Post
pbelkner
post Feb 7 2011, 15:20
Post #87





Group: Members
Posts: 412
Joined: 13-June 10
Member No.: 81467



QUOTE (kode54 @ Feb 7 2011, 15:06) *
/5*2.

It's the way lib1770 is doing it as well, cf. "bs1770_stats.c", lines 66, 67.
Go to the top of the page
+Quote Post
benski
post Feb 7 2011, 16:20
Post #88


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



QUOTE (Raiden @ Feb 7 2011, 00:36) *
It is doing window overlap. Otherwise, the block energies from the test in post 81 would be different, and not all 0.654412.

Oh ok I see what it's doing now. It's doing the windowing in ebur128_calc_gating_block instead of ebur128_add_frames_/ebur128_filter_. Sorry for making such a stink, just wanted to be sure that the library was working correctly. I had gotten concerned when it was running the filter on 200ms blocks but now I see that it's doing the sum-of-squares on 400ms segments. Sorry for the alarm smile.gif
Go to the top of the page
+Quote Post
nucelar
post Feb 7 2011, 16:59
Post #89





Group: Members
Posts: 6
Joined: 3-February 11
Member No.: 87885



Hi raiden, you're last improvements/ additions are awesome!
Good work!
Go to the top of the page
+Quote Post
Raiden
post Feb 20 2011, 18:55
Post #90





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



Hi again,
I've uploaded 0.2.0:

Library:
- Removed the function ebur128_set_channel_map. Use ebur128_set_channel instead.
- Fixed the samplerate bug (the library would not work correctly with a samplerate that was not divisible by 5).
- ebur128_*_multiple functions skip NULL pointers.

Scanners:
- Removed the Python tagging script. Tagging is now directly integrated into the scanners. I'm using TagLib for that. Usage example:
CODE
r128-sndfile -t album foo/ (tags files in the folder "foo" as an album)
r128-sndfile -t album -r bar/ (recursively tag sub-folders of "bar", each sub-folder as an album)
r128-sndfile -t album foo.ogg bar.ogg baz.ogg (tag files as an album)
r128-sndfile -t track foo.ogg bar.ogg baz.ogg

This should now provide much of the functionality of similar commandline tools like vorbisgain or mp3gain.

Download
Go to the top of the page
+Quote Post
habasud
post Feb 23 2011, 09:46
Post #91





Group: Members
Posts: 15
Joined: 22-February 11
Member No.: 88387



QUOTE (Raiden @ Feb 20 2011, 18:55) *
Hi again,
I've uploaded 0.2.0:
[...]


Hi Raiden,

first of all thanks for making and publishing such a useful peace of software! What a lucky day it was when I saw it compiling on FreeBSD without errors... smile.gif

With the latest version I noticed that a script I wrote for normalizing didn't work any more. It turned out that the text output of r128-ffmpeg changed somewhat. You can see it when you pipe the output to e.g. hexdump. So in my case piping the output to bc for calculation causes a syntax error.

And a suggestion: As I said I use your software for calculation of normalization. When it turns out that an audiofile has to be made "louder" I measure the given headroom with sox - actualy another time costing scan. Maybe it would be useful if you add a switch that outputs the highest true-peak within a given file.

This is just meant as a feedback - thank you!
Go to the top of the page
+Quote Post
Raiden
post Feb 23 2011, 12:38
Post #92





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



QUOTE (habasud @ Feb 23 2011, 09:46) *
It turned out that the text output of r128-ffmpeg changed somewhat. You can see it when you pipe the output to e.g. hexdump. So in my case piping the output to bc for calculation causes a syntax error.

Yes, the output changed in the last version a bit... Currently it is not really meant for scripts as it writes everything to stderr. I will try to write something useful to stdout.
QUOTE (habasud @ Feb 23 2011, 09:46) *
And a suggestion: As I said I use your software for calculation of normalization. When it turns out that an audiofile has to be made "louder" I measure the given headroom with sox - actualy another time costing scan. Maybe it would be useful if you add a switch that outputs the highest true-peak within a given file.

I will try to get true peak measurement done for the next version, thanks for the feedback. smile.gif
Go to the top of the page
+Quote Post
C.R.Helmrich
post Feb 26 2011, 13:10
Post #93





Group: Developer
Posts: 686
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



FYI: EBU tech doc 3343 has been published.

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
Raiden
post Feb 26 2011, 20:20
Post #94





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



I've uploaded 0.2.1.

Library:
- removed the ebur128_start_new_segment and ebur128_loudness_segment functions. To calculate track/album values use one library state for each track, then calculate the album gain with ebur128_loudness_global_multiple.
- documented ebur128.h with Doxygen.

Scanner:
- fixed a crash with the FFmpeg scanner.
- fixed recursive folder scanning (didn't work in 0.2.0).
- implemented true peak scanning! Those values will NOT be used for the ReplayGain tags.
- improved scanner output a lot. Hopefully it is now easier to parse by scripts.

The scanner can now calculate all three values recommended by the EBU: "Programme Loudness", "Loudness Range" and "True Peak Level".
QUOTE
$ ./r128-sndfile -l -p true ~/music/bad\ loop\ -\ Luo/*.flac

-12.81 LUFS, LRA: 14.16 LU, true peak: 0.99826229, /home/jan/music/bad loop - Luo/bad loop - Luo - 01 Nio.flac
-11.15 LUFS, LRA: 8.26 LU, true peak: 0.99095666, /home/jan/music/bad loop - Luo/bad loop - Luo - 02 Eri Valeire.flac
-10.14 LUFS, LRA: 11.79 LU, true peak: 0.99171823, /home/jan/music/bad loop - Luo/bad loop - Luo - 03 Kauniit Ihmiset.flac
-11.31 LUFS, LRA: 11.75 LU, true peak: 0.92898595, /home/jan/music/bad loop - Luo/bad loop - Luo - 04 Mmin.flac
-26.13 LUFS, LRA: 14.87 LU, true peak: 0.25203928, /home/jan/music/bad loop - Luo/bad loop - Luo - 05 3b Or T.flac
-14.10 LUFS, LRA: 11.40 LU, true peak: 1.02603507, /home/jan/music/bad loop - Luo/bad loop - Luo - 06 Kannas Nsp.flac
--------------------------------------------------------------------------------
-11.75 LUFS, LRA: 13.34 LU, true peak: 1.02603507


A note regarding true peak scanning: Even with 4x oversampling it is still possible to underread the "true" peak value by ~0.6 dB. The EBU therefore recommends a "Maximum Permitted True Peak Level" of -1dBTP (1 dB below 0dBFS).

Download
Go to the top of the page
+Quote Post
Raiden
post Feb 27 2011, 22:40
Post #95





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



And here is 0.2.2, with a single change:
- added "--gate" option to specify the relative gate. The standard uses a gate of -8 dB, but will switch to -10 dB in the next revision. This option is for testing only and will be removed when the standard is updated!

I've scanned my FLAC library again with the new gate:
CODE
r128-sndfile --gate -8.0 *.flac: -10.52 LUFS
r128-sndfile --gate -10.0 *.flac: -10.77 LUFS

-> a difference of 0.25 dB.
Go to the top of the page
+Quote Post
nucelar
post Mar 3 2011, 18:01
Post #96





Group: Members
Posts: 6
Joined: 3-February 11
Member No.: 87885



QUOTE (Raiden @ Feb 27 2011, 22:40) *
And here is 0.2.2, with a single change:
- added "--gate" option to specify the relative gate. The standard uses a gate of -8 dB, but will switch to -10 dB in the next revision. This option is for testing only and will be removed when the standard is updated!


Hey Raiden you are great!!! Not only are you on the cutting edge of the spec but also do it all well. FYI I've tested libebur128 with the test set provided by Qualis Audio
http://www.qualisaudio.com/downloads.htm
and ALL test have passed perfectly!!! (except test #2 which is ac-3 encoded 5.1 and I'm not sure if libebur128 can handle this)
Just a couple of remarks:
-It should be sufficient to indicate 1 decimal point in dB figures. For example, some test that should measure -23.0 LUFS according to Qualis, measure -22,97 LUFS in libebur. Of course it is the same but it just doesn't look so good smile.gif And I think EBU Tech docs use 1 decimal point always.
- Please make the True Peak measurement in dBTP!!! At first I thought the measurement was wrong (0,793..) but of course this is linear for -2.0 dBTP !! Good work!!!

On the other hand:
Could someone guide me through the steps of how to build an executable for OSX from source code? is it complicated? Does it imply modifying the code? I have XCode (mainly because of some bundled apps) but I have no idea where to start tongue.gif
Best regards!
Carles
Go to the top of the page
+Quote Post
Raiden
post Mar 3 2011, 22:33
Post #97





Group: Developer
Posts: 224
Joined: 14-September 04
Member No.: 17002



QUOTE (nucelar @ Mar 3 2011, 18:01) *
Hey Raiden you are great!!! Not only are you on the cutting edge of the spec but also do it all well. FYI I've tested libebur128 with the test set provided by Qualis Audio
http://www.qualisaudio.com/downloads.htm
and ALL test have passed perfectly!!! (except test #2 which is ac-3 encoded 5.1 and I'm not sure if libebur128 can handle this)
I'm glad that it works for you!
I've had a quick look at LM2.wav, and it seems to be just a normal stereo 24-bit PCM wave file at 48000 Hz. They say in TechNote #2:
QUOTE
This signal is Dolby Digital encoded and
stimulates all channels simultaneously, including
the LFE.
I don't have any idea what they mean.
Also, does anyone know why they use a 75% overlap instead of a 50% overlap?

QUOTE (nucelar @ Mar 3 2011, 18:01) *
-It should be sufficient to indicate 1 decimal point in dB figures. For example, some test that should measure -23.0 LUFS according to Qualis, measure -22,97 LUFS in libebur. Of course it is the same but it just doesn't look so good smile.gif And I think EBU Tech docs use 1 decimal point always.
You're right. I've been using two digit precision because it's a bit more helpful in debugging. smile.gif
QUOTE (nucelar @ Mar 3 2011, 18:01) *
- Please make the True Peak measurement in dBTP!!! At first I thought the measurement was wrong (0,793..) but of course this is linear for -2.0 dBTP !! Good work!!!
I will add an option for displaying the peak in dBTP.

QUOTE (nucelar @ Mar 3 2011, 18:01) *
Could someone guide me through the steps of how to build an executable for OSX from source code? is it complicated? Does it imply modifying the code? I have XCode (mainly because of some bundled apps) but I have no idea where to start tongue.gif
I don't know much about OSX, but you'll probably need to use some kind of ports system like MacPorts.
A command like "sudo port install libsndfile taglib cmake glib2" should install all needed dependencies to build the r128-sndfile scanner with the instructions in the README.
Go to the top of the page
+Quote Post
nucelar
post Mar 4 2011, 11:53
Post #98





Group: Members
Posts: 6
Joined: 3-February 11
Member No.: 87885



Hi!
Thanks for the Macports suggestion, i will try it.
File #2 is most probably a Dolby-E encoded 5.1 signal. Dolby-E is a broadcast codec that compresses and encodes 5.1 to a stereo bitstream. Decoding needs expensive hardware or software. I wouldn't care about this.
Cheers
Go to the top of the page
+Quote Post
habasud
post Mar 4 2011, 12:36
Post #99





Group: Members
Posts: 15
Joined: 22-February 11
Member No.: 88387



...maybe someone finds it useful to visualize the output of r128-ffmpeg (-m/-s/-i). I wrote a simple shellscript that passes the data to gnuplot:
http://home.arcor.de/claus.misfeldt/shared/r128_plot.tar.gz

Thank you Raiden for great work!
Go to the top of the page
+Quote Post
Surfi
post Mar 5 2011, 18:37
Post #100





Group: Members
Posts: 175
Joined: 1-October 04
Member No.: 17420



::

SSE2 Win32 compile doesn't work with command
"r128-sndfile.exe" -p true *.flac

OS is Win XP SP3 with all updates. Non-SSE build has no problems.


Greetings ...

::
Go to the top of the page
+Quote Post

8 Pages V  « < 2 3 4 5 6 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 31st July 2014 - 03:09