Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Wavegain problems in Linux (Read 18303 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Wavegain problems in Linux

Hi guys,

Just starting off using Linux (Debian) for burning mix CDs.  In the olden days, I used the outstanding Burrrn program.  It would wavegain the files and then burn them.  Now I am trying to replicate the same process on Linux.

Since there is no Burrrn for Linux, I need to do the decoding and wavegaining manually.

Just installed the Wavegain package from rarewares.org's Debian repository.  Running Wavegain seems to work fine when analyzing WAV files (ignore the values here because I was analyzing files that had already been MP3gained prior to decoding to WAV):

Code: [Select]
andy@homer:/mnt/hdf2/tmp$ wavegain -l ./*.wav
  -1.76 dB |  15525 |  0.82 |    12677 |    0  |    1  | ./01 - Joel Plaskett - Happen Now.wav
  -1.18 dB |  12803 |  0.87 |    11176 |    9  |    7  | ./02 - Blackalicious - Powers.wav
  -1.32 dB |  12088 |  0.86 |    10383 |  -3  |    -3  | ./03 - My Morning Jacket - Off the Record (edit).wav.wav
  -0.86 dB |  12238 |  0.91 |    11084 |  -1  |    -1  | ./04 - North Mississippi Allstars - Bang Bang Lulu.wav
  -1.15 dB |  12144 |  0.88 |    10638 |  -1  |    0  | ./05 - Paul McCartney - Fine Line.wav
  -1.57 dB |  13212 |  0.83 |    11027 |    0  |    0  | ./06 - Madvillain - Money Folder (Four Tet Remix).wav
  -0.80 dB |  11234 |  0.91 |    10245 |    0  |    0  | ./07 - Gang Of Four - To Hell With Poverty.wav
  -1.61 dB |  13724 |  0.83 |    11402 |  -1  |    -2  | ./08 - British Sea Power - It Ended On An Oily Stage.wav
  -0.76 dB |  12906 |  0.92 |    11824 |    0  |    0  | ./09 - Kathleen Edwards - Summerlong.wav
  -1.00 dB |  12903 |  0.89 |    11499 |    0  |    0  | ./10 - R. Kelly - Happy Summertime (Feat. Snoop Dogg).wav
  -1.43 dB |  17662 |  0.85 |    14981 |  -41  |  -63  | ./11 - Vitalic - Trahison.wav
  -1.32 dB |  13638 |  0.86 |    11715 |    1  |    1  | ./12 - Black Rebel Motorcycle Club - Ain't No Easy Way.wav
  -1.48 dB |  12796 |  0.84 |    10791 |    0  |    -1  | ./13 - Teenage Fanclub - Fallen Leaves.wav
  -0.64 dB |  10800 |  0.93 |    10033 |    1  |    1  | ./14 - The Go-Betweens - Lavender.wav
  -0.59 dB |  19959 |  0.93 |    18649 |    0  |    0  | ./15 - Ry Cooder - Los Chucos Suaves.wav
  -1.57 dB |  15306 |  0.83 |    12775 |    0  |    2  | ./16 - Danger Doom - The Mask feat. Ghostface.wav
  -0.98 dB |  12224 |  0.89 |    10919 |    0  |    0  | ./17 - Franz Ferdinand - Do You Want To.wav
  -0.31 dB |  11831 |  0.96 |    11416 |  -1  |    0  | ./18 - Aimee Mann - She Really Wants You.wav
  -1.21 dB |  12654 |  0.87 |    11008 |  -1  |    -4  | ./19 - Big Star - Lady Sweet.wav
  -1.14 dB |  10757 |  0.88 |    9434 |  -1  |    0  | ./20 - The Rolling Stones - Rough Justice.wav
  -0.38 dB |  14416 |  0.96 |    13798 |  12  |    12  | ./21 - Kanye West - Heard 'Em Say (Feat. Adam Levine of Maroon 5).wav
  -0.68 dB |  14473 |  0.92 |    13383 |    2  |    2  | ./22 - Devendra Banhart - I Feel Like A Child.wav

 Recommended Album Gain:  +0.32 dB      Scale: 1.0375


 WaveGain Processing completed normally
*** glibc detected *** free(): invalid next size (fast): 0x08074050 ***
Aborted

When it comes time to apply the analysis though it doesn't seem to work properly though.  The process starts and nothing happens.  Looking at the process in top shows the program using 100% of the CPU power, but nothing appears to be happening.  Does Wavegain really take this long on Linux?  Or am I missing something?  Any suggestions?


Wavegain problems in Linux

Reply #2
Yeah, that message seems strange to me as well.  Odd that it appears during the analysis (which seems to work fine), but there is no such message when trying to apply the wavegain.

Looking at package lists available related to "glibc", it appears that I may have to check and make sure that they're all installed.  Maybe that will help.

Wavegain problems in Linux

Reply #3
Quote
> *** glibc detected *** free(): invalid next size (fast): 0x08074050 ***Aborted

Something fishy must be going on.
[{POST_SNAPBACK}][/a]


That means you free an invalid pointer. [a href="http://www.valgrind.org]Valgrind[/url] is your friend to see where is the real problem for such cases.
An eye for eye will make the whole world blind

Wavegain problems in Linux

Reply #4
Download the wavegain 1.2.5 sources from rarewares, apply the following patch:
Code: [Select]
diff -u wavegain_c/audio.c wavegain/audio.c
--- wavegain_c/audio.c  2005-09-16 12:46:00.000000000 +0200
+++ wavegain/audio.c    2005-10-27 18:51:26.000000000 +0200
@@ -30,7 +30,6 @@
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
-#include <io.h>
#include <fcntl.h>
#include <malloc.h>
#include <math.h>
Common subdirectories: wavegain_c/WaveGain and wavegain/WaveGain
diff -u wavegain_c/wavegain.c wavegain/wavegain.c
--- wavegain_c/wavegain.c       2005-09-16 12:59:58.000000000 +0200
+++ wavegain/wavegain.c 2005-10-27 18:51:43.000000000 +0200
@@ -21,7 +21,6 @@
#include <math.h>
#include <string.h>
#include <ctype.h>
-#include <io.h>
#include <fcntl.h>
#include <malloc.h>
#include "gain_analysis.h"

And compile with "gcc *.c -o wavegain -O2". The resultant binary appears to work as advertised.

Wavegain problems in Linux

Reply #5
weird - i'll look at it when i get home (and update the package...)

thanx
mike

Wavegain problems in Linux

Reply #6
<gentle nudge>

Hey Mike, did you get a chance to look at the wavegain package yet?

Thanks, Andy

Wavegain problems in Linux

Reply #7
ohoh... mmm very much forgot about it actually - thanks for the nudge (i can be reached faster by email, btw)

it is repackaged at RW/Debian

actually, the version at RW/Debian was 1.2.1 which i had moved the reference of io.h to sys/io.h (common for win32->linux - i'm assuming this caused the problem above)

here is the 1.2.5 patch (with Makefile, debian dir excluded):
Code: [Select]
diff -N wavegain-1.2.5.orig/audio.c wavegain-1.2.5/audio.c
33d32
< #include <io.h>
Common subdirectories: wavegain-1.2.5.orig/debian and wavegain-1.2.5/debian
diff -N wavegain-1.2.5.orig/Makefile wavegain-1.2.5/Makefile
0a1,11
> DESTDIR=`pwd`/debian/wavegain
>
> all:
>       gcc *.c -o wavegain -lm -lsndfile
>
> install:
>       install -d $(DESTDIR)/usr/bin/
>       install -m 0755 wavegain $(DESTDIR)/usr/bin/
>
> clean:
>       rm -Rf wavegain *.o
Common subdirectories: wavegain-1.2.5.orig/WaveGain and wavegain-1.2.5/WaveGain
diff -N wavegain-1.2.5.orig/wavegain.c wavegain-1.2.5/wavegain.c
24d23
< #include <io.h>


cabbagerizzle - thanx for the patch

goweropolis - let me know if things aren't kosher

RW/debian is updated:
http://rarewares.org/debian.html

Wavegain problems in Linux

Reply #8
Thanks xmixahlx,

I updated to the latest WaveGain (v1.2.5 (0.34) Compiled Nov  4 2005.).  I ran on a couple of files to test:
Code: [Select]
andy@homer:/mnt/hdf2/tmp$ wavegain ./*.wav
 -9.57 dB |  32767 |  0.33 |    10888 |    0  |     0  | ./01 - E-Pro.wav
 -9.22 dB |  32767 |  0.35 |    11335 |    0  |     1  | ./02 - Que Onda Guero.wav
*** glibc detected *** double free or corruption (out): 0x08074090 ***
Aborted


Works fine, but again a weird error message at the end.

So I run and choose to apply radio gain:
Code: [Select]
andy@homer:/mnt/hdf2/tmp$ wavegain -y ./*.wav
 -9.57 dB |  32767 |  0.33 |    10888 |    0  |     0  | ./01 - E-Pro.wav
 -9.22 dB |  32767 |  0.35 |    11335 |    0  |     1  | ./02 - Que Onda Guero.wav
*** glibc detected *** double free or corruption (out): 0x08074090 ***
0.3322770.345939Aborted


It looks like it worked, but still a weird error message.  I run the analysis again to double check the application of the gain values.

Code: [Select]
andy@homer:/mnt/hdf2/tmp$ wavegain ./*.wav
 -9.57 dB |  32767 |  0.33 |    10888 |    0  |     0  | ./01 - E-Pro.wav
 -9.22 dB |  32767 |  0.35 |    11335 |    0  |     1  | ./02 - Que Onda Guero.wav
*** glibc detected *** double free or corruption (out): 0x08074090 ***
Aborted


It doesn't appear to have altered the files at all.

If there's any more information I can provide to assist, please let me know.  Thanks.

Wavegain problems in Linux

Reply #9
Quote
Code: [Select]
andy@homer:/mnt/hdf2/tmp$ wavegain ./*.wav
*** glibc detected *** double free or corruption (out): 0x08074090 ***
Aborted

[a href="index.php?act=findpost&pid=339659"][{POST_SNAPBACK}][/a]


Run this in your shell or put it in your .profile or equivalent.

  export MALLOC_CHECK_=0

This disables the check for memory errors in glibc.
Hopefully this will work as a temporary solution.

Wavegain problems in Linux

Reply #10
Or the real fix (which could be found after running valgrind and analysing, taking 10 minutes in all):

In main.c in static FILE_LIST* alloc_node(const char* file)

change
Code: [Select]
node->dc_offset = calloc(2, sizeof(double *));

to
Code: [Select]
node->dc_offset = calloc(10, sizeof(double));


We want doubles not pointers to doubles. I changed to 2 to 10, as you would get probs if you want to open multi channel wavs. I think 10 is pretty safe but not a real fix. It should in fact be set to the source's channels.

Then I found out wavegain is leaking, and a few minutes more I fixed it. I think I should submit a patch to John. (In wavegain.c in get_gain: Search for "float **buffer = malloc(sizeof(float *) * wg_opts->channels);" and the second find is the place where to substitue with this code. I guess you'll see how.)

Code: [Select]
    else
    {
 float **buffer = malloc(sizeof(float *) * wg_opts->channels);
 
 for (i = 0; i < wg_opts->channels; i++)
     buffer[i] = malloc(BUFFER_LEN * sizeof(float));
 
 while (1) {
     long samples_read;

     samples_read = wg_opts->read_samples(wg_opts->readdata, buffer, BUFFER_LEN, 0, 0);

     if (samples_read == 0) {
   break;
     }
     else {
   if (samples_read < 0) {
       /* Error in the stream. Not a problem, just reporting it in case
     * we (the app) cares. In this case, we don't
     */
   }
   else {
       long i;
       int j;

       for (i = 0; i < wg_opts->channels; i++) {
     for (j = 0; j < samples_read; j++) {
         if (buffer[i][j] > 0.f)
       total_plus[i] += buffer[i][j];
         else if (buffer[i][j] < 0.f)
       total_minus[i] += buffer[i][j];
         buffer[i][j] *= 0x7fff;
         if (FABS(buffer[i][j]) > peak)
       peak = FABS(buffer[i][j]);
     }
       }

       if (AnalyzeSamples(buffer[0], buffer[1], samples_read,
         wg_opts->channels) != GAIN_ANALYSIS_OK) {
     fprintf(stderr, " Error processing samples.\n");
     for (i = 0; i < wg_opts->channels; i++)
         if (buffer[i]) free(buffer[i]);
     if (buffer) free(buffer);

     goto exit;
       }
   }
     }
 }
 for (i = 0; i < wg_opts->channels; i++)
     if (buffer[i]) free(buffer[i]);
 if (buffer) free(buffer);


BTW, wavegain uses an old version of gain_analysis.c, as far as I can see. You may want to sync to latest one found in latest mp3gain for a nice speed-up.


I also wrote a tiny cmake (cmake.org) project file. Using cmake you can compile cross platform. With this project file you can generate unix makefiles, kdevelop and msvc++ project files (and probably more).

CMakeLists.txt
Code: [Select]
# cmake project file by Prakash Punnoor 
PROJECT(Wavegain)

ADD_EXECUTABLE(wavegain
              audio.c
              dither.c
              gain_analysis.c
              getopt1.c
              getopt.c
              main.c
              misc.c
              recurse.c
              wavegain.c
)
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-lm ")

INSTALL_TARGETS(/bin wavegain)


It is not perfect, but works. To compile, run something like:

Code: [Select]
cmake . -DCMAKE_C_FLAGS:STRING="-march=i686 -O2"
make


Of course you should use optimal CFLAGS for your platform.

HTH,

Prakash


Wavegain problems in Linux

Reply #12
Quote
Could you take a look at LAME's RG scanning to see if it could benefit similarly, please?

BTW, foobar2000's RG scanning is fast too.
[a href="index.php?act=findpost&pid=339679"][{POST_SNAPBACK}][/a]


The one in lame cvs looks recent (or even newer). Perhaps somebody should check which is latest/fastest. fb2k'sdk should include its gain_analysis file, I'd guess.

Wavegain problems in Linux

Reply #13
Quote
In main.c in static FILE_LIST* alloc_node(const char* file)

change
Code: [Select]
node->dc_offset = calloc(2, sizeof(double *));

to
Code: [Select]
node->dc_offset = calloc(10, sizeof(double));

We want doubles not pointers to doubles. I changed to 2 to 10, as you would get probs if you want to open multi channel wavs. I think 10 is pretty safe but not a real fix. It should in fact be set to the source's channels.
Thanks, there was an issue there. I've changed the declaration to two doubles rather than use a pointer since the algorithm only supports either one, or two, channels, and not multichannel.
Quote
Then I found out wavegain is leaking, and a few minutes more I fixed it. I think I should submit a patch to John. (In wavegain.c in get_gain: Search for "float **buffer = malloc(sizeof(float *) * wg_opts->channels);" and the second find is the place where to substitue with this code. I guess you'll see how.)
Actually, the leak was in write_gains, not get_gain, but thanks for making me look at the code!! 
Quote
BTW, wavegain uses an old version of gain_analysis.c, as far as I can see. You may want to sync to latest one found in latest mp3gain for a nice speed-up.
I've incorporated the files from the mp3gain source which, incidentally, required moving from floats to doubles - probably no bad thing. 

Thanks again for the interest and advice. I've one or two other things to look at before I make another release.

Wavegain problems in Linux

Reply #14
Quote
Actually, the leak was in write_gains, not get_gain, but thanks for making me look at the code!!


Well, I haven't actually ever used writing so I didn't find that leak, but trust me, there is a leak in get_gain as well. 

Quote
Thanks again for the interest and advice. I've one or two other things to look at before I make another release.


Glad I could help.  Have you thought a about adding a project file? Currenlty there is nothing for Linux/Unix building. If you never tried cmake, try it.  It even features a GUI if you want.

Something unrelated: Why don't you try to get this stuff integrated into mp3gain? Then the porblem of syncing with gain_analysis would be gone and you'd have an all-in-one tool. aac managed to get integrated...

Oh some last advise: Though using mem on the stack ("static arrays") is easier to handle (like you changed above issue with dc_offset), if you write beyond the border, you have a very hard time debugging, as tools like valgrind won't detect it. So unless it is performance critical, try to use mem on the heap ("dynamic /malloced arrays") as much as possible. It is better to leak than to crash. In this case usually the leak can be located easier than crashing due to corrupting the stack.

Wavegain problems in Linux

Reply #15
Quote
Thanks again for the interest and advice. I've one or two other things to look at before I make another release.
[a href="index.php?act=findpost&pid=339821"][{POST_SNAPBACK}][/a]

As long as you're still looking at things, you might consider what I suggested yesterday in the Lame thread:  adding a small bias into the yulewalk filter to speed up processing of digital silence.  E.g.:

Code: [Select]
rompel@helium:~/osrc/wavgain-1.2.5$ diff -bu gain_analysis_orig_c gain_analysis.c
--- gain_analysis_orig_c        Fri Sep 16 12:51:14 2005
+++ gain_analysis.c     Sat Nov  5 13:38:26 2005
@@ -201,7 +201,7 @@
    size_t  k;

    for ( i = 0; i < nSamples; i++ ) {
-        y = input[i] * b[0];
+        y = 1e-15 + input[i] * b[0];
        for ( k = 1; k <= order; k++ )
            y += input[i-k] * b[k] - output[i-k] * a[k];
        output[i] = (Float_t)y;

Wavegain problems in Linux

Reply #16
Quote
Quote
Thanks again for the interest and advice. I've one or two other things to look at before I make another release.
[a href="index.php?act=findpost&pid=339821"][{POST_SNAPBACK}][/a]

As long as you're still looking at things, you might consider what I suggested yesterday in the Lame thread:  adding a small bias into the yulewalk filter to speed up processing of digital silence.[a href="index.php?act=findpost&pid=339858"][{POST_SNAPBACK}][/a]

Actually that code doesn't exist in the current version.

Wavegain problems in Linux

Reply #17
Quote
Actually that code doesn't exist in the current version.
[a href="index.php?act=findpost&pid=339863"][{POST_SNAPBACK}][/a]
Picky, picky

Here's what I did in Lame, which should be similar to whatever mp3gain code you're using:

Code: [Select]
rompel@helium:~/osrc/lame-3.97/libmp3lame$ diff -U 6 gain_analysis{-orig,}.c
--- gain_analysis-orig.c        Sat Apr 24 10:47:03 2004
+++ gain_analysis.c     Fri Nov  4 16:57:19 2005
@@ -142,12 +142,13 @@
filterYule (const Float_t* input, Float_t* output, size_t nSamples, const Float_t* kernel)
{
    /*register double  y;*/

    while (nSamples--) {
        *output =  input [0]  * kernel[0]
+        + 1e-10
         - output[-1] * kernel[1]
         + input [-1] * kernel[2]
         - output[-2] * kernel[3]
         + input [-2] * kernel[4]
         - output[-3] * kernel[5]
         + input [-3] * kernel[6]

Wavegain problems in Linux

Reply #18
@rompel

Out of interest: Could you explain why this is faster? I don't understand but want to.

Wavegain problems in Linux

Reply #19
Quote
Out of interest: Could you explain why this is faster? I don't understand but want to.
[{POST_SNAPBACK}][/a]

The problem I'm trying to fix is one where a section of digital silence either in the middle or at the end of an input file causes a large slowdown in the Replaygain calculation (as was pointed out [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38480&hl=]here[/url]). 

When a stable IIR filter, such as the two at the heart of the Replaygain algorithm, is initialized with non-zero inputs then fed a long series of zero inputs, it exhibits exponentially decaying output.  In a mathematically perfect world, this output series would decay indefinitely, approaching but never quite reaching zero.  However, using floating-point arithmetic, what happens is that the series eventually approaches the limit of what can be represented in floating-point, there is a loss of precision, and the series no longer decays.  Sometimes it jumps to zero, but very often it gets stuck repeating one or several very small numbers.

That alone would not be a problem for Replaygain except for the fact that these very small numbers will be subnormals on any floating-point architecture that supports them.  While subnormals, and the gradual underflow they enable, have many advantages, there is a long history of slow implementation.  It would seem that most modern CPUs handle subnormals in microcode (which, slow as it is, beats the early SPARCs which handled them in software!).  How slow are they?  I've read that for the Pentium III and similar CPUs there is a 50-150 clock cycle penalty for subnormal operands (compared to 3 cycles for a FP add or 5 cycles for a FP multiply).

So if subnormals are bad, how do we avoid them?  If you are using the SSE floating-point unit, there's a bit you can set to avoid them entirely.  But that's a big if, plus I don't know of any portable way in C to set that bit even if you can establish that you're on an SSE-capable machine.  Another way is to clamp small values to zero, (e.g. something like x = fabs(x)<1e30 ? 0.0 : x).  This method, unfortunately, involves quite a bit of overhead.  Another alternative in the case of Replaygain would be to look for long strings of zero input and treat them as a special case.  Although this would require quite a bit of new code and some overhead, it would be a lot less opaque than the method I chose.

That method is to purposely introduce a small error to make subnormals highly unlikely.  By adding a small constant at every stage of the filter calculation, I am biasing the output by a slightly larger, but still very small, amount.  This will have negligible impact on the gain calculation.  But, with this change, the only way to get a subnormal is if the unbiased filter output were of almost exactly the same magnitude and opposite sign.  This might occur as a transient condition, but it won't happen as a steady state.  Rather given a string of zero inputs, the output will converge to some small normal floating-point number.  Besides being a trivial change in the code, the performance impact should be negligible since the additional addition can be overlapped with the multiplies on modern CPUs.

I hope this explanation is useful.

--John

Wavegain problems in Linux

Reply #20
Thanx! Though I am no DSP guy and as such didn't understand your explanation reagrding that; I think I understood the technical things to a bigger extent. And now I understand why a reverb implementation called "freeverb" gets rid of unnormalised values by setting them to zero (though freeverb talks of denorms).

Wavegain problems in Linux

Reply #21
Quote
Thanx! Though I am no DSP guy and as such didn't understand your explanation reagrding that; I think I understood the technical things to a bigger extent. And now I understand why a reverb implementation called "freeverb" gets rid of unnormalised values by setting them to zero (though freeverb talks of denorms).
[a href="index.php?act=findpost&pid=340096"][{POST_SNAPBACK}][/a]

Yeah, denormals and subnormals are the same thing.

Wavegain problems in Linux

Reply #22
john(33) - any word on when a updated wavegain will be released?

apologies if this has been mentioned already.


later

 

Wavegain problems in Linux

Reply #23
Quote
john(33) - any word on when a updated wavegain will be released?

apologies if this has been mentioned already.


later
[a href="index.php?act=findpost&pid=340307"][{POST_SNAPBACK}][/a]

Sorry for the delay, I'm just in the process of rebuilding the system after adding a 250GB drive, repartitioning and generally messing with everything!!!  I'll get back to it within the next day, or two.

Wavegain problems in Linux

Reply #24
Quote
john(33) - any word on when a updated wavegain will be released?

apologies if this has been mentioned already.


later
[a href="index.php?act=findpost&pid=340307"][{POST_SNAPBACK}][/a]

I've uploaded a 1.2.6beta (win32 executable only) to Rarewares. This version uses the most recent GainAnalysis algorithm, includes Tycho's addition's (thanks  ), adds an option to use an Album DC Offset correction in place of the default Track DC Offset correction and , finally, includes a little code tidying up.

As this is beta, please report any prooblems/bugs/errors and I'll sort them as soon as possible. Thanks for everyones interest and help.