IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Reply to this topicStart new topic
strange results using low-pass filter ( lame 3.98/3.99.5 )
paracent
post Aug 14 2013, 20:42
Post #1





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



Hello,

For encoder(s) testing purpose I created a 19.3kHz >>tone<< , 48kHz, 10 sec sample wav file., using 'sox'.
After a number of different tests for quality/sound reproduction ( doing on aac he/h2 & ogg ) I decided to test it on mp3 using lame 3.98 & 3.99.5
All samples were imported into 'baudline' to build a spectrogram. ( total about 50 )
when I got to lame, I set the low-pass filter to 19.5, 20, 21, and 22kHz, after importing the 20 and lower low-pass samples into baudline, my first words were: WTF?

here is the original WAV: ( all just 19.3khz tone )


this is a 48kHz / 21kHz low-pass MP3: ( lame )


this is the, same sample, but 19.5kHz low-pass ( lame ) ( this is when I asked WTF? )



in all test cases for OGG / AAC / MP3 I encoded to the specific format, and back to WAV - to make sure the results are constant, using ffmpeg to reencode )
even if I do not re-encode back to wav, the results are the same on LAME for low-pass lower, or equal to 20kHz.

now, this is just one tone 19.3kHz... Imagine, all the music collections converted to MP3... using lo-pass 20kHz or lower.

The tests I am doing are about the encoder accuracy, and of course, if the accuracy is higher, quality of the encoded file will be also better.
So far, the results I got, put WAV, FLAC and OGG as the most accurate, closest to the original sample formats. ( various samples, including single, high freq. tone(s) )

None of other formats I have been working on shows such "strange" behavior...

If anybody has an explanation, to my "WTF" question, I would really appreciate....

Mike

edit:
used the following LAME opts:

lame -V0 --lowpass 19.5 output.wav output_v0_lp19.5_99lame.mp3
and
lame -V0 --lowpass 21 output.wav output_v0_lp21_99lame.mp3


This post has been edited by paracent: Aug 14 2013, 20:48
Go to the top of the page
+Quote Post
saratoga
post Aug 14 2013, 21:26
Post #2





Group: Members
Posts: 4868
Joined: 2-September 02
Member No.: 3264



Clipping? If you use a tone at -6dB does it work normally?
Go to the top of the page
+Quote Post
paracent
post Aug 14 2013, 21:35
Post #3





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



well, this is not about clipping...

look at the last screenshot - you should see only 19.3kHz there, what about the rest of tones that "appeared from nowhere" after using LAME, lo-pass =<20kHz
this is the issue I am trying to find an explanation...

Thanks!



QUOTE (saratoga @ Aug 14 2013, 16:26) *
Clipping? If you use a tone at -6dB does it work normally?

Go to the top of the page
+Quote Post
saratoga
post Aug 14 2013, 21:41
Post #4





Group: Members
Posts: 4868
Joined: 2-September 02
Member No.: 3264



QUOTE (paracent @ Aug 14 2013, 16:35) *
look at the last screenshot - you should see only 19.3kHz there,


Under the assumption that there is no clipping, yes. Does it clip?
Go to the top of the page
+Quote Post
mixminus1
post Aug 14 2013, 21:50
Post #5





Group: Members
Posts: 688
Joined: 23-February 05
Member No.: 20097



QUOTE
Imagine, all the music collections converted to MP3... using lo-pass 20kHz or lower

I'm not imagining very many music collections that have a substantial amount of material at a 48 kHz sampling rate (to say nothing of 19 kHz content @ 0 dBFS) - try your experiment at 44.1 kHz...

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...and as I implied above, the artifacts are greatly reduced at 44.1 kHz @ 0 dBFS.


--------------------
"Not sure what the question is, but the answer is probably no."
Go to the top of the page
+Quote Post
paracent
post Aug 14 2013, 21:52
Post #6





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:

n_samples: 480431
mean_volume: -3.1 dB
max_volume: 0.0 dB
histogram_0db: 135966

>no clipping< under no assumptions - I do not do all these tests assuming anything...



QUOTE (saratoga @ Aug 14 2013, 16:41) *
QUOTE (paracent @ Aug 14 2013, 16:35) *
look at the last screenshot - you should see only 19.3kHz there,


Under the assumption that there is no clipping, yes. Does it clip?

Go to the top of the page
+Quote Post
DonP
post Aug 14 2013, 21:53
Post #7





Group: Members (Donating)
Posts: 1471
Joined: 11-February 03
From: Vermont
Member No.: 4955



QUOTE (saratoga @ Aug 14 2013, 16:26) *
Clipping? If you use a tone at -6dB does it work normally?


Shouldn't clipping just produce harmonics of the fundamental tone? Still a useful question.
Go to the top of the page
+Quote Post
db1989
post Aug 14 2013, 22:49
Post #8





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (paracent @ Aug 14 2013, 21:52) *
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:
QUOTE
>no clipping< under no assumptions - I do not do all these tests assuming anything...
But you should. Lossy encoding can introduce clipping via the creation of inter-sample overs, which does not require clipping in the original file but is especially favoured by the latter having a lot of samples close to 0 dBFS. You should assume that.
Go to the top of the page
+Quote Post
saratoga
post Aug 14 2013, 23:06
Post #9





Group: Members
Posts: 4868
Joined: 2-September 02
Member No.: 3264



QUOTE (paracent @ Aug 14 2013, 16:52) *
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:


If it reaches 0dB (that is, full scale) then its certainly clipped. You need to lower the level.

QUOTE (mixminus1 @ Aug 14 2013, 16:50) *
Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...


Ha, i was right. Good guess.
Go to the top of the page
+Quote Post
greynol
post Aug 14 2013, 23:49
Post #10





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Yet another example of someone not understanding lossy compression and attempting to second-guess the encoder by adding additional unnecessary switches.

QUOTE (paracent @ Aug 14 2013, 12:42) *
For encoder(s) testing purpose...

Any results derived from these tests will be completely useless unless the entire purpose is to ABX these specific tones, in which case you should be using ABX tools, not frequency analysis.

This post has been edited by greynol: Aug 19 2013, 17:12


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
paracent
post Aug 15 2013, 00:05
Post #11





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



Did more tests..
Well, without lowering by -3dB,
with 20.1kHz lo-pass there is no change... still looking bad, same result as 20kHz
with 20.2kHz lo-pass it looks much better, the "foreign" frequencies go up to max -95 ~ -100dB

- without dropping to -3dB, would a lo-pass filter of 0.1kHz higher eliminate the artifacts completely? ( or to -95dB max )?
this is what happened, and this is also my original "wtf" question.




QUOTE (saratoga @ Aug 14 2013, 18:06) *
QUOTE (paracent @ Aug 14 2013, 16:52) *
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:


If it reaches 0dB (that is, full scale) then its certainly clipped. You need to lower the level.

QUOTE (mixminus1 @ Aug 14 2013, 16:50) *
Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...


Ha, i was right. Good guess.


This post has been edited by paracent: Aug 15 2013, 00:06
Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 00:11
Post #12





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (paracent @ Aug 14 2013, 16:05) *
looks

This is the problem, full stop.

We listen with our ears, not our eyes. How the signal "looks" is irrelevant.

Brush up on TOS #8 before posting on this topic again.

This post has been edited by greynol: Aug 15 2013, 00:13


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
paracent
post Aug 15 2013, 00:30
Post #13





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



I guess a better place to ask this question would be the lame mailing list, sorry for skipping the TOS MOD.
on the other hand, this information will prevent me from trusting my ears today, and tomorrow, if I decide to use a different codec, I will find out what mistake I made in the first place, but this might be already too late... I guess I am not the only one that found himself in such position.
Of course, TOS, either agree or gtfo... regardless on what you got to say - reminds me on something (...)



QUOTE (greynol @ Aug 14 2013, 19:11) *
QUOTE (paracent @ Aug 14 2013, 16:05) *
looks

This is the problem, full stop.

We listen with our ears, not our eyes. How the signal "looks" is irrelevant.

Brush up on TOS #8 before posting on this topic again.


This post has been edited by paracent: Aug 15 2013, 00:31
Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 00:33
Post #14





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (paracent @ Aug 14 2013, 16:30) *
TOS, either agree or gtfo
Pretty much.

QUOTE
regardless on what you got to say
If it isn't compliant with our terms we aren't interested.


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
db1989
post Aug 15 2013, 00:36
Post #15





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (paracent @ Aug 15 2013, 00:30) *
on the other hand, this information will prevent me from trusting my ears today
Why? Do you think your eyes know how to hear better than your ears do? Can you hear a difference? If not, what you see is irrelevant.

QUOTE
and tomorrow, if I decide to use a different codec, I will find out what mistake I made in the first place, but this might be already too late...
What do you mean?

QUOTE
Of course, TOS, either agree or gtfo... regardless on what you got to say - reminds me on something (...)
Reminds you of what? Come out and say it. Being passive-aggressive by implying through ellipsis isn’t going to get you anywhere.

[edit]

To be clear, I at least am not saying you’re not allowed to talk about observations that aren’t audible, or that this is desirable behaviour for an encoder, but those things are very different from concluding that the effect is audibly problematic without actually performing a valid test for audibility. And because we’re overwhelmingly interested in things that are audible, we tend not to spend much time worrying about the visual effects of lossy compression, which have no correlation with its effects upon the ears.

Personally, I’d be most interested in a discussion of whether this is unavoidable in MP3/LAME, rather than worrying about its implications for normal music; I’m sure we’d have heard all about those if they existed.

This post has been edited by db1989: Aug 15 2013, 00:43
Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 00:40
Post #16





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (db1989 @ Aug 14 2013, 16:36) *
implying through ellipsis isnít going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
Propheticus
post Aug 15 2013, 00:42
Post #17





Group: Members
Posts: 219
Joined: 10-September 11
Member No.: 93615



If you actually read tos 8 you'd see it's there for a good reason. This forum is full of discussions about this subject too.

As to this 'problem': Just lower your input a bit, as suggested to you but you stubbornly refused to do. Going from sources at 0dB to mp3 can add clipping. Just lower to -3dB and check again.
Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 00:47
Post #18





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



FWIW, I don't think this is resulting from clipping. Feel free to generate the samples and have a look for yourself.


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
Makaki
post Aug 15 2013, 00:57
Post #19





Group: Members
Posts: 59
Joined: 20-May 13
Member No.: 108227



This discussion would be a lot more useful if someone added, at least a quick ABX of said samples. It would be an interesting Visual observation if it is followed by the hearing observation.
Being a single tone, I don't think the ABX would take that long.

PS: Personally, I can't hear a 19.3kHz tone.

This post has been edited by Makaki: Aug 15 2013, 00:59
Go to the top of the page
+Quote Post
greynol
post Aug 15 2013, 01:14
Post #20





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance" then don't "lame.exe --useless --additional --switches --that --cripple --performance".


--------------------
Placebophiles: put up or shut up!
Go to the top of the page
+Quote Post
paracent
post Aug 15 2013, 05:39
Post #21





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



QUOTE (greynol @ Aug 14 2013, 19:40) *
QUOTE (db1989 @ Aug 14 2013, 16:36) *
implying through ellipsis isnít going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


misuse? so you telling me now what am I allowed to listen to?

QUOTE
If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance"


is this a quote that you just apply to anything? is using a 20/21 kHz lo-pass a 'misuse' of switches? I see... are you really using 16.5kHz lo pass for all your music needs? - this would explain...

I re-encoded a couple of samples to about 300 result files, used 5 codecs, different options for ogg, aac... and the only mess, that is not only visible, but sounds terrible, is when lame is used in a certain condition...

It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age. That is why I came here, and started this topic, and not to fight with Gargamel. In any case, your avatar says it all biggrin.gif

Go to the top of the page
+Quote Post
saratoga
post Aug 15 2013, 07:32
Post #22





Group: Members
Posts: 4868
Joined: 2-September 02
Member No.: 3264



QUOTE (paracent @ Aug 15 2013, 00:39) *
QUOTE (greynol @ Aug 14 2013, 19:40) *
QUOTE (db1989 @ Aug 14 2013, 16:36) *
implying through ellipsis isnít going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


misuse?


MP3 isn't made to compress clipped synthetic signals. You're supposed to give it some headroom, or else use replaygain. If you don't, the results will distort.

I'm surprised they're so badly distorted here, but yes, you're doing it wrong smile.gif

QUOTE (paracent @ Aug 15 2013, 00:39) *
so you telling me now what am I allowed to listen to?


No one has said anything like that.

QUOTE (paracent @ Aug 15 2013, 00:39) *
It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age.


If you're not willing to accept the answers to questions, you should probably not ask them in the first place. Its just a waste of everyone's time to argue with you in that case.
Go to the top of the page
+Quote Post
[JAZ]
post Aug 15 2013, 12:00
Post #23





Group: Members
Posts: 1767
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



QUOTE (paracent @ Aug 15 2013, 01:05) *
- without dropping to -3dB, would a lo-pass filter of 0.1kHz higher eliminate the artifacts completely?


If you don't pay attention to what you do, and don't listen to people that know how things work, then, you're expected to get invalid results.
Now, do the test again and read what LAME is telling you in the command line about the exact lowpass frequency it is using, and not what you tell it to use.

Next, I did the same test with LAME 3.99.5 (windows, 32bit rarewares build) -V0, (stereo, 48Khz 19.3Khz 0dBFS, filtered with --lowpass 19.5, with a 16bit and a 24bit file), and in all cases, the noise was below 90dBs, much below than your case.


QUOTE (greynol @ Aug 15 2013, 02:14) *
If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance" then don't "lame.exe --useless --additional --switches --that --cripple --performance".


Sure he's changing the defaults, and who knows what more is he doing since I can't replicate it, but the change is not that far from defaults. I.e. Using -V1 instead of -V0 already uses a filter in that range.




QUOTE (paracent @ Aug 15 2013, 06:39) *
It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age. That is why I came here, and started this topic, and not to fight with Gargamel. In any case, your avatar says it all biggrin.gif


If you had, for a moment, the thought that Hydrogenaudio is a place where we simply sit down and accept whatever "dogma" is established, you should learn a bit more about us before continue.

Said that, what we have learned in these years is to not blindly believe in what a spectogram shows when talking about lossy compression, and to avoid focusing on artificial samples when testing the reliability of these codecs.

What you found might or might not be a misfunction of the filter, but so far, my results don't show the same than yours.




Edit: Ok, I found why my results were so different, and definitely verifies what saratoga said from the beginning.
I opened the MP3s directly in Audacity, and they opened in floating precision. The sines go above full scale but since I had it in float, they didn't clip and the noise was maintained below 90dBFS.

Why is there such a difference? Well, there's a small problem with the filter, because it generates some aliasing/mirroring (i.e in the spectogram, there's a line at 19.3 and another at 19.7, around 24dBs softer).

This post has been edited by [JAZ]: Aug 15 2013, 12:48
Go to the top of the page
+Quote Post
db1989
post Aug 15 2013, 12:55
Post #24





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE ([JAZ] @ Aug 15 2013, 12:00) *
Edit: Ok, I found why my results were so different, and definitely verifies what saratoga said from the beginning.
I opened the MP3s directly in Audacity, and they opened in floating precision. The sines go above full scale but since I had it in float, they didn't clip and the noise was maintained below 90dBFS.

Why is there such a difference? Well, there's a small problem with the filter, because it generates some aliasing/mirroring (i.e in the spectogram, there's a line at 19.3 and another at 19.7, around 24dBs softer).
Fantastic. Now we can start getting somewhere. Thanks for the explanation!
Go to the top of the page
+Quote Post
paracent
post Aug 15 2013, 13:17
Post #25





Group: Members
Posts: 13
Joined: 14-August 13
Member No.: 109616



perfect, "start getting somewhere" sounds great ( even if it is distorted ). a bumpy road, maybe a couple of wrong expressions I used, but I am glad, that there is an explanation other than "-3dB". Problems are found on "per case basis", they often can be "hidden" similar to this case, otherwise Windoze should never show a BSOD. Suggesting workarounds in some cases is acceptable, and often an only way, but I did not start the topis to hear workarounds... If there is a confirmed bug/issue with Lame, regardless from the comments on how "improper" my encoding attempts are, I will be more than happy that this topic was started. Don't you agree? Thanks again!
Mike

[edit]

this is the source wav and two mp3 files, at 20.1 and 20.2kHz lo-pass

original wav
20.1kHz lo pass mp3
20.2kHz lo pass mp3

thanks!

This post has been edited by paracent: Aug 15 2013, 13:45
Go to the top of the page
+Quote Post

3 Pages V   1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st August 2014 - 03:17