IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Reply to this topicStart new topic
Interesting Histograms.
Woodinville
post May 29 2011, 00:24
Post #1





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



I've been doing histograms of audio stuff, as well as calculating spectral flatness measure, so I can reproduce some of the information I had back when I was starting on the PAC/NBC/AAC work.

I've found an album that a fair number of missong codes.

I've also found an album that has a max under +-16384 and has over 75% missing codes inside of that.

I didn't originally set up to calculate ratio of missing codes to all codes. But I think I shall have to.

I'll put up some photos of the various histograms when I get a bit more done.

Wow, batman, doesn't anyone look at this stuff before pressing!?

This post has been edited by Woodinville: May 29 2011, 00:43


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 03:32
Post #2





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



http://s238.photobucket.com/albums/ff228/j...l%20Histograms/

Some histograms. My ghu is octave SLOW.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 04:33
Post #3





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



ETA:

octave wavread returns numbers 1 level out of 32768 below -1 and and exactly +1 on the positive side.

Does anybody have a WORKING wavread fer bleep's sake? Anyone? Help? wavread should never go below -1, or above 1 - 2^-(nbits-1)

Also, try this in the latest octave

round(-.0001)
ans=-0

Yep, minus zero.

I guess I'll spring for my own personal copy of ratlab and signal processing package, that's intolerable.

So, all of these histograms are "almost right", it's pretty clear that they are right except that they might be missing more missing codes, since one can't count on the actual wavread scaling.

Bleepity blip bleep blip.

Your in annoyance

jj

This post has been edited by Woodinville: May 29 2011, 04:54


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 07:12
Post #4





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



Ok, if you're using wavread from octave, be sure to change the constants for 16 and 24 bit files from 2^(nbits-1)-1 to 2^(nbits-1)

Otherwise you waste a lot of time getting incorrect results.

Plots above will be changing as they get done. Expletive.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
C.R.Helmrich
post May 29 2011, 09:30
Post #5





Group: Developer
Posts: 686
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



Thanks for debugging Octave's wavread, JJ!

Hmm, looking at the Robert Lee (piano) histogram, it looks like someone, as the last step before pressing, applied a positive-dB gain on the 16-bit data? blink.gif

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 09:47
Post #6





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (C.R.Helmrich @ May 29 2011, 01:30) *
Thanks for debugging Octave's wavread, JJ!

Hmm, looking at the Robert Lee (piano) histogram, it looks like someone, as the last step before pressing, applied a positive-dB gain on the 16-bit data? blink.gif

Chris


It's a nearly-homemade recording of a really good stride pianist from the middle of Indiana.

If you want to hear youtube-quality, listen to "Allen Dale" on Youtube. But try to ignore it when he talks politics. Listen to 12th St Rag, and just watch the hands fly.

Oh, and you know, I'm not under any IP restrictions right now, I could post the histogram script for Octave if anyone gave two whits. It's not like it's even remotely close to rocket science, I think anyone here could write it in a half-hour as long as you also fix the wavread constant. &*(&( that was irritating. Oh, and the Spectral Flatness Measure is for 2048 Hann Window. Green Day and Avril Lavigne now up, Chumbawama coming up soon.

This post has been edited by Woodinville: May 29 2011, 09:49


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
xnor
post May 29 2011, 11:47
Post #7





Group: Developer
Posts: 490
Joined: 29-April 11
From: Austria
Member No.: 90198



Interesting plots, thanks.

QUOTE (Woodinville @ May 29 2011, 01:24) *
I've also found an album that has a max under +-16384 and has over 75% missing codes inside of that.


So it's basically a 15-bit album? laugh.gif

I'm interested in the script.

This post has been edited by xnor: May 29 2011, 11:49
Go to the top of the page
+Quote Post
romor
post May 29 2011, 16:25
Post #8





Group: Members
Posts: 668
Joined: 16-January 09
Member No.: 65630



QUOTE (Woodinville @ May 29 2011, 08:12) *
Ok, if you're using wavread from octave, be sure to change the constants for 16 and 24 bit files from 2^(nbits-1)-1 to 2^(nbits-1)

Otherwise you waste a lot of time getting incorrect results.

Plots above will be changing as they get done. Expletive.

Excuse my ignorance and if I understand you right, but why should Octave (or anything else) scale normalized data on 2^(n-1)-1 instead 2^(n-1)?


--------------------
scripts: http://goo.gl/M1qVLQ
Go to the top of the page
+Quote Post
Paulhoff
post May 29 2011, 17:44
Post #9





Group: Members
Posts: 106
Joined: 3-June 05
From: Coconut Creek Fl
Member No.: 22486



QUOTE (Woodinville @ May 28 2011, 19:24) *
I've been doing histograms of audio stuff, as well as calculating spectral flatness measure, so I can reproduce some of the information I had back when I was starting on the PAC/NBC/AAC work.

I've found an album that a fair number of missong codes.

I've also found an album that has a max under +-16384 and has over 75% missing codes inside of that.

I didn't originally set up to calculate ratio of missing codes to all codes. But I think I shall have to.

I'll put up some photos of the various histograms when I get a bit more done.

Wow, batman, doesn't anyone look at this stuff before pressing!?


I don't have anything, and I'm not at all sure what you want to begin with.


My Bad???

Paul

smile.gif smile.gif smile.gif


--------------------
"Reality is merely an illusion, albeit a very persistent one." Albert Einstein
Go to the top of the page
+Quote Post
xnor
post May 29 2011, 18:17
Post #10





Group: Developer
Posts: 490
Joined: 29-April 11
From: Austria
Member No.: 90198



QUOTE (romor @ May 29 2011, 17:25) *
Excuse my ignorance and if I understand you right, but why should Octave (or anything else) scale normalized data on 2^(n-1)-1 instead 2^(n-1)?


because 2^15-1 = 32767 decimal = 7FFF hex = highest possible value for a signed, 16-bit integer

and -32768 decimal = 8000 hex

see two's complement

This post has been edited by xnor: May 29 2011, 18:22
Go to the top of the page
+Quote Post
romor
post May 29 2011, 18:39
Post #11





Group: Members
Posts: 668
Joined: 16-January 09
Member No.: 65630



If the problem is about histogram function, and that's why the need for +1 bin (number of samples + 1), maybe that can make some sense to me

Data presented by wavread from Octave is same as text output by Cool Edit Pro, or output from libsndfile library on my PC

This post has been edited by romor: May 29 2011, 18:39


--------------------
scripts: http://goo.gl/M1qVLQ
Go to the top of the page
+Quote Post
Axon
post May 29 2011, 19:36
Post #12





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



OT:

OH MY FUCKING GOD JJ you read my mind about octave.

I tried my *best* to like it a month ago, when I attempted to use it for "real" audio work. I really did. Right before I discovered
  • wavread does not do blocked WAV input, only file-at-a-time. I actually got most of the way through rewriting this, but
  • Whoever wrote wavread does not comprehend two's complement arithmetic, and maps [-32768, 32767] to [-1,1]. That's right. Try reading a 16-bit WAV of all zeroes in. Can't do it! The samples will read as -2^-17. (EDIT: Ah yes you saw this too)
  • wavread has not been materially updated since 2001. Or maybe 1996. Or possibly 1993. Needless to say it has not gone through a process of continuous improvement
  • Octave itself requires upwards of 4 seconds on my P4 to populate a 128k sample array with the evaluation of a window function. Which is approximately 3 orders of magnitude slower than it ought to be.


At which point I concluded that: it is plainly obvious that nobody actually does useful audio processing work in Octave, and it would likely be a waste of my time to attempt to do so, rather than look at other free numerical processing solutions. (I'm thinking mostly about F# nowadays.)

Then I posted "GNU Octave fucking blows" on Facebook. Now see jj, if I friended you, you would have seen that comment, and you wouldn't have had to go through all that trouble, eh wink.gif

This post has been edited by Axon: May 29 2011, 20:38
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 20:15
Post #13





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (romor @ May 29 2011, 08:25) *
QUOTE (Woodinville @ May 29 2011, 08:12) *
Ok, if you're using wavread from octave, be sure to change the constants for 16 and 24 bit files from 2^(nbits-1)-1 to 2^(nbits-1)

Otherwise you waste a lot of time getting incorrect results.

Plots above will be changing as they get done. Expletive.

Excuse my ignorance and if I understand you right, but why should Octave (or anything else) scale normalized data on 2^(n-1)-1 instead 2^(n-1)?


I don' t know, but that's what it did. !?

You can go into the library and change it to the proper 2^(n-1) without much trouble.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Axon
post May 29 2011, 20:18
Post #14





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



QUOTE (Woodinville @ May 29 2011, 14:15) *
I don' t know, but that's what it did. !?

You can go into the library and change it to the proper 2^(n-1) without much trouble.


I vaguely remember doing this too.

Yet another reason to give Octave the Viking funeral it richly deserves, by dousing it with lighter fluid, and setting it on fire.
Go to the top of the page
+Quote Post
Axon
post May 29 2011, 20:52
Post #15





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



QUOTE (Woodinville @ May 28 2011, 18:24) *
I've also found an album that has a max under +-16384 and has over 75% missing codes inside of that.

I didn't originally set up to calculate ratio of missing codes to all codes. But I think I shall have to.

In some meaningful sense, couldn't this be considered a "13-bit" recording, rather than a "15-bit" recording?

And I suppose, more generally: when we are evaluating coding efficiency, are we "literally" evaluating Komolgorov complexity, and using that as an estimator for sound quality?
Go to the top of the page
+Quote Post
xnor
post May 29 2011, 20:56
Post #16





Group: Developer
Posts: 490
Joined: 29-April 11
From: Austria
Member No.: 90198



Forgive my previous reply, I didn't read properly.

in the Octave 3.4.0 sources, wavread.m, line 201 onward:

CODE
    ## Normalize samples.
    switch (bits_per_sample)
      case 8
        yi = (yi - 128)/127;
      case 16
        yi /= 32767;
      case 24
        yi /= 8388607;
      case 32
        yi /= 2147483647;


This seems to be the problem you're talking of and should be corrected to 32768 for 16-bit for example.

Or patch it to don't normalize at all.

This post has been edited by xnor: May 29 2011, 20:58
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 21:00
Post #17





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (xnor @ May 29 2011, 12:56) *
Forgive my previous reply, I didn't read properly.

in the Octave 3.4.0 sources, wavread.m, line 201 onward:

CODE
    ## Normalize samples.
    switch (bits_per_sample)
      case 8
        yi = (yi - 128)/127;
      case 16
        yi /= 32767;
      case 24
        yi /= 8388607;
      case 32
        yi /= 2147483647;


This seems to be the problem you're talking of and should be corrected to 32768 for 16-bit for example.

Or patch it to don't normalize at all.


I patched it to 32768 and 2147483648

Then I could get the results I expected on a file that was half -max and half +max without using wrong multipliers smile.gif

I think 8-bit is mulaw, and if I'm right, that's just totally wrong, but I'm not entirely sure. Been a while since I used 8-bit.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 21:02
Post #18





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (xnor @ May 29 2011, 03:47) *
Interesting plots, thanks.

QUOTE (Woodinville @ May 29 2011, 01:24) *
I've also found an album that has a max under +-16384 and has over 75% missing codes inside of that.


So it's basically a 15-bit album? laugh.gif

I'm interested in the script.


Actually, as Axon points out, it's 13 bits, because there are so many missing levels, as well as it running under half.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
xnor
post May 29 2011, 21:10
Post #19





Group: Developer
Posts: 490
Joined: 29-April 11
From: Austria
Member No.: 90198



QUOTE (Woodinville @ May 29 2011, 22:00) *
I think 8-bit is mulaw, and if I'm right, that's just totally wrong, but I'm not entirely sure. Been a while since I used 8-bit.


Nah it's just that 8-bit PCM is unsigned so the range of values is 0 to 255 with a centerpoint of 128.
So if you substract 128 you get the same result you'd get with signed 8-bit integers (-128 to 127). Of course the division by 127 should be corrected to 128 like in the other cases.

edit: oh you already filed a bug, great!

This post has been edited by xnor: May 29 2011, 21:20
Go to the top of the page
+Quote Post
Axon
post May 29 2011, 21:23
Post #20





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



If anybody's interested, I still have my mostly-complete-and-working rewrite of wavread to implement buffered WAV file reading (along with other mucho bug fixes of course).

EDIT: Ah, I saw jj's bug too. I guess I should stop being so antisocial and submit a few of my own too.

This post has been edited by Axon: May 29 2011, 21:32
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 21:28
Post #21





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Axon @ May 29 2011, 13:23) *
If anybody's interested, I still have my mostly-complete-and-working rewrite of wavread to implement buffered WAV file reading (along with other mucho bug fixes of course).


Well, I'd be interested if it was working.

smile.gif

I suppose I should just attach the histogram/sfm calculation here.

Nothing fancy, I assure you.

--- this is freeware, nothing is guaranteed at all---
CODE
clear all
close all
clc

fname='02.wav';
x=wavread(fname);
x=x';

len=length(x)

his(1:65536)=0;

low=32768;
high=32768;
windd=hann(2048)';

sfmmean=0;
nmeas=0;

for ii=1:len
for jj=1:2

t=x(jj,ii);
if ( t < -1)
t=-1
fflush(stdout);
end
if (t >65535/65536)
t=65535/65536
fflush(stdout);
end

t=round(t*32768+32769);


his(t)=his(t)+1;
if (t < low)
low=t;
end
if (t > high)
high=t;
end
end
if (mod(ii,1024)==0)
if (ii>1024)
for kk=1:2
w=x(1,ii-2047:ii);
w=w .* windd;
wt=fft(w);
wt= wt .* conj(wt);
wt=max(wt,.0000000001);
am=sum(wt)/2048;
gm= exp (sum(log(wt))/2048);
sfm=gm/am;
if ( am > .5)
nmeas=nmeas+1;
sfm=10*log10(sfm);
sfmmean=sfmmean+sfm;
end


end
end
if (mod(ii,1024*16)==0 )
ii/len
fflush(stdout);
end
end

end

tot=sum(his);
his=his/tot;
his=max(his, .000000000001);
xax=-32768:32767;

semilogy(xax,his);
axis([-40000 40000 1e-10 1e-1]);
nmeas=nmeas
if (nmeas >0)
sfmmean=sfmmean/nmeas
end
low=low-32769
high=high-32769
fname

---end of freeware---

Change fname as you will smile.gif

Its slow. The time is spent not in the fft part, but in the very simple histogram part, too. No idea why. It's slower than a snail glued to a rock.

This post has been edited by Canar: May 30 2011, 07:18
Reason for edit: Now with more [codebox]


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Axon
post May 29 2011, 21:52
Post #22





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



Mods, could one of you please add *.m as an allowed upload extension?
Go to the top of the page
+Quote Post
Canar
post May 29 2011, 22:34
Post #23





Group: Super Moderator
Posts: 3348
Joined: 26-July 02
From: princegeorge.ca
Member No.: 2796



I'll bug the admins. I can't do this. In the interim, zip the .m and post that?


--------------------
You cannot ABX the rustling of jimmies.
No mouse? No problem.
Go to the top of the page
+Quote Post
googlebot
post May 29 2011, 22:34
Post #24





Group: Members
Posts: 698
Joined: 6-March 10
Member No.: 78779



QUOTE (Axon @ May 29 2011, 20:36) *
Then I posted "GNU Octave fucking blows" on Facebook. Now see jj, if I friended you, you would have seen that comment, and you wouldn't have had to go through all that trouble, eh wink.gif


I somewhat associate 'Axon' with wooing Woodinville for quite some time now, but I think that's a new level. smile.gif
Go to the top of the page
+Quote Post
Woodinville
post May 29 2011, 23:41
Post #25





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Canar @ May 29 2011, 14:34) *
I'll bug the admins. I can't do this. In the interim, zip the .m and post that?


Well, screen-copying the text above into octave works like a champ smile.gif

It's not like there's any special character stuff in it.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post

3 Pages V   1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th July 2014 - 01:07