Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Measuring audio quality degradation (Read 14396 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Measuring audio quality degradation

Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

My question is how one can determine the "best" track out of many tracks having different bitrates or/and codecs?

For instance 192Kbit MP3 track produced using a CDDA source is definitely better than another one which is a reencode from 128Kbit MP3 to 320Kbit MP3 just for the sake of "Yo-ho, we have 320KBit tracks here".

Another possible source of audios are reencodes from YouTube lousy audio into yet another codec (usually 320KBit mp3).

Is there a way to compare different musical files and say conclusively that this or that one has the highest quality? I.e. the least amount of distortions and the widest dynamic range.


Measuring audio quality degradation

Reply #1
Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

My question is how one can determine the "best" track out of many tracks having different bitrates or/and codecs?

For instance 192Kbit MP3 track produced using a CDDA source is definitely better than another one which is a reencode from 128Kbit MP3 to 320Kbit MP3 just for the sake of "Yo-ho, we have 320KBit tracks here".

Another possible source of audios are reencodes from YouTube lousy audio into yet another codec (usually 320KBit mp3).

Is there a way to compare different musical files and say conclusively that this or that one has the highest quality? I.e. the least amount of distortions and the widest dynamic range.



(1) Become a trained listener

(2) Exercise that gift.

Measuring audio quality degradation

Reply #2
What about measuring that digitally?

Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.

Measuring audio quality degradation

Reply #3
Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.


Most people can't. And if you can't tell a difference by listening, why would you care?

Measuring audio quality degradation

Reply #4
Because I need to tell which one of two low quality encodes is "better". ;-)

Measuring audio quality degradation

Reply #5
What is "better"?

Measuring audio quality degradation

Reply #6
What about measuring that digitally?


I lecture people all day about how well technical tests inform us about sound quality, but to this day there does not appear to be a generally accepted 100% reliable way to do that with perceptually coded files.

Oh, technical tests can help to a degree - for example if a 44/16  file has an obvious brick wall filter @ 15-18 KHz we probably can guess how it got there and there's a perceptual coder in the story.

Quote
Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.


Most people can't ever do it, and even those who sometimes can, most of that lucky group can't do it 100% of the time.

To some degree this is like a discussion as to whether a certain person is using certain err, clothes items to augment their appearance...  ;-)

 

Measuring audio quality degradation

Reply #7
Is there a way to compare different musical files and say conclusively that this or that one has the highest quality?

Yes, a double blind AB test, perhaps a forced choice test.

Quote
I.e. the least amount of distortions and the widest dynamic range.

No.  Are you familiar with any mp3 encoder that compromises dynamic range at the bitrates you mentioned? I'm not; and I doubt such an encoder exists. "Distortions" is too vague, but assuming you're talking about audible artifacts, you're asking for an algorithm that knows something about psychoacoustics. Read the first part of my post again to see what is a proper way to judge on that metric.

Measuring audio quality degradation

Reply #8
I lecture people all day about how well technical tests inform us about sound quality, but to this day there does not appear to be a generally accepted 100% reliable way to do that with perceptually coded files.
...and most of the methods that do exist require access to both the original (perfect, lossless) version as well as the messed up (lossy) version, to estimate how bad the messed up version sounds. They can't compare two messed up versions and tell you which one is more faithful to an unknown original, or which one sounds "better".

Cheers,
David.

Measuring audio quality degradation

Reply #9
Better does not necessarily mean most faithful to the reference.  In order to have a preference you don't need access to the reference at all.

Trivial case in point: .5 dB amplification vs 1 dB amplification when neither results in audible distortion.

What if the listener likes the sound of artifacts specific to a particular encoding?

Measuring audio quality degradation

Reply #10
Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.


Most people can't. And if you can't tell a difference by listening, why would you care?


Exactly. The best version of a given track is the one you like the most, not the one that's measures a certain way that is determined to be "best".

Subjective listening tests are often (rightfully) ridiculed, but for finding which track you like best, only your own subjective choice matters.

Measuring audio quality degradation

Reply #11
Better does not necessarily mean most faithful to the reference.
I know, that's why I put "or". And I put better in quotes.

I was just saying: the automated methods that do exist can't answer the OP's question, even if you're willing to overlook their inferiority to human ears in the tasks they are designed for.

Some people did look at blind quality assessment (i.e. you don't have the original, but how bad is the version that you do have); I think they got somewhere with telephone quality speech, but I don't know if they got anywhere with music.

Cheers,
David.

Measuring audio quality degradation

Reply #12
...then let's try it again.

They can't compare two messed up versions and tell you [...] which one sounds "better".

Yes, they most certainly can; with or without the use of quotes.

Measuring audio quality degradation

Reply #13
...and most of the methods that do exist require access to both the original (perfect, lossless) version as well as the messed up (lossy) version, to estimate how bad the messed up version sounds. They can't compare two messed up versions and tell you which one is more faithful to an unknown original, or which one sounds "better".

Cheers,
David.


Exactly what I'm talking about.

Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable or appropriate but it seems like its my only choice.

BTW, I'm curious - auCDtect shows such characteristics as "Coefficient of nonlinearity of a phase", "First order smothness" and "Second order smothness"  but so far I haven't been able to identify a pattern in these highly cryptic numbers. Does anyone know what they mean and how they can be applied in this situation?

Measuring audio quality degradation

Reply #14
Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable or appropriate but it seems like its my only choice.
To add to the problem of choosing a version, multiple versions of the same track don't necessarily have to be based on the same source material, or master. It's easy and common to mis-attribute mastering differences to audio codecs (and settings), if one doesn't realize this issue. So two versions of the same song might not only differ in encoder settings and choice, but also in the source material. In such a case, there is really no other meaningful way to choose the song you like best other than trusting your senses, even if it turns out it's the WMA version with the lowest bitrate.
It's only audiophile if it's inconvenient.

Measuring audio quality degradation

Reply #15
Of course it should be possible to develop an algorithm that ranks signals in an order which highly rank-correlates with subjective quality.  That is precisely what the psy-models of lossy encoders do - however they do, naturally, assume access to the lossless reference. Measuring artifacts should also be possible, and some are available: click/pop detection used for vinyl rips, for example, depend upon detecting "something bad" with certain reliability. And someone on this forum pointed out to me (when I asked a releated question) that in video processing, there are artifact removal filters - of course with side effects yes, but if they succeed in presenting a subjectively "better" output, then they are beyond the level differentiating by quality.

Building an algorithm to rank signals, would likely take the same job as the development of psy models: listen, model, and tune (i.e. estimate) the model to mimic human preferences. It shouldn't be hard - assuming you can be satisfied with less than perfection - but it could be very time consuming to do the job.

Measuring audio quality degradation

Reply #16
...then let's try it again.

They can't compare two messed up versions and tell you [...] which one sounds "better".

Yes, they most certainly can; with or without the use of quotes.
Ah, I see what you mean.

Arnie was talking about "technical tests" in the original quote, not human ears. I was talking about the better ear-like technical tests (PEAQ etc).

Measuring audio quality degradation

Reply #17
Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable
Why not? The lossy compression used on YouTube and most other places is designed to destroy the signal as much as possible while fooling human ears that almost nothing has been changed. It's not trying to please an oscilloscope or a spectrogram or a computer - it's trying to please your ears.

Maintaining the measurable signal integrity and quality at a very high level would also please your ears - but it would not let YouTube squeeze something that sounds half-decent into 128kbps or less.

A high quality mp3 has about 20-30dB Signal to Noise Ratio (SNR) - i.e. the added noise is always about 25dB below the music. A cassette tape without Dolby has about 45dB SNR - i.e. the noise from the cassette tape is measurably lower - it measures much better than the mp3. Yet the hiss on the tape is easily audible, while the noise in the mp3 is mostly inaudible. That's because the noise in the mp3 is hidden in areas your ears are insensitive to (it is masked by the music), while the noise on the tape isn't.

There are similar differences between different encoders and formats. Some add more noise (so measure worse) but hide it better (as far as human ears are concerned) so sound better.

The absolute best automatic measurements of lossy audio encoding quality try to mimic human ears - but they can't do it perfectly. That's why the gold standard for assessing the quality of audio codecs is a carefully controlled listening test using human listeners. (Ditto video codecs and viewing tests).

Cheers,
David.

Measuring audio quality degradation

Reply #18
Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

In my experience with illegally downloaded music, the bigger quality factor is how badly someone altered the music believing they're making it better. I've heard music that was severely EQ'd and overly compressed. So for me, I'd rather have a 128 kbps version of a file taken off a CD, than a 320 kbps version that someone messed with (or recorded off vinyl).

--Ethan
I believe in Truth, Justice, and the Scientific Method

Measuring audio quality degradation

Reply #19
It sounds like you want a computer program to tell you which is 'better'. This is better than you actually listening?

When I find problems with my audio system I get out the test equipment to try to find and correct the problem which was originally discovered with my ears. Trust me that your ears are more sensitive than the best test gear but the equipment is getting better.


Measuring audio quality degradation

Reply #20
Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

In my experience with illegally downloaded music, the bigger quality factor is how badly someone altered the music believing they're making it better. I've heard music that was severely EQ'd and overly compressed. So for me, I'd rather have a 128 kbps version of a file taken off a CD, than a 320 kbps version that someone messed with (or recorded off vinyl).

--Ethan


Oh, yeah, that's been my experience as well - 320Kb mp3s that sound a lot worse than 128Kbit due to normalization or/and compression.

Measuring audio quality degradation

Reply #21
Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable
Why not?


It sounds like you want a computer program to tell you which is 'better'. This is better than you actually listening?




Well first of all a good algorithm saves human labour. If you have a million files to test, then I would argue that an algorithm with a 0.75 rank correlation to your ears, is a better practical solution than listening to all of them.

Second, a statistical model may be way more consistent than human judgement. Obviously if you are judging for others; four ears hear better than two, and a model built on many ears' experience could be way superior to my pair of ears if the purpose is to be a proxy for your pair of ears. (There is even nothing that says it isn't a superior model of your ears as of tomorrow either, if you are having a bad ear day.)

Third, for lossy encoding and artifact removal - e.g. pops/clicks - the algorithm must know what is "better". Your ears can tell that "this second without that click" is an improvement, and that "this lossy file is good enough", but in order to actually get that, you need an algorithm that knows what to replace that split second of signal with, resp. an algorithm with a decent psy model. Otherwise you have to either do sound engineering manually, or let the software generate an enormous lot of more or less random suggestions ... back to the human labour argument, then.



So yes, I would very much like to have computer program to tell me what is better. And luckily there are such ones around for certain purposes.

Measuring audio quality degradation

Reply #22
If you have a million files to test, then I would argue that an algorithm with a 0.75 rank correlation to your ears, is a better practical solution than listening to all of them.
Humans tend to listen in real time at best, so with strict deadlines and huge amounts of data it's more practical to use software to control audio quality. One of the professional applications used by major content providers is Interra's Baton. It's a file based QC system. Advantage over human listening is speed and consistency. When I inquired a few years ago the price tag was about $30k and afaik there isn't a cheaper version available. It can do audio and video QC. The audio part features:
Quote
• Silence • Clipping • Mute • Test tones • Phase detection • Wow & Flutter • Audio distortion • Jitter • Transient noise • High Frequency noise • Background noise • Minimum Audio Level • Audio Dropout • Over Modulation Noise • Click and pop • Loudness compliance (ITU, EBU, CALM Act)
I can imagine that for media exchange between content providers this can be a great system. At least you're not depending on a good or bad day of an audio engineer who has to listen to (often) boring material
@Glenn: don't you work in a pro A/V environment ? Do you use automated QC ?

Measuring audio quality degradation

Reply #23
Currently the best available tool known to me is PEAQ and it's quite terrible. Does not have enough knowledge of human psychoacoustics...
It's not even 75% accurate for lightly psychoacoustically lossy compressed audio, such as Wavpack lossy mode, much less for more advanced lossy codecs.
It will underrate frequency response errors and channel balance as well, making it quite not useful for QC on, say, headphones. It cannot deal with reverberation (most of the time) and many nonlinearities adequately either.

Now then, feel free to improve it with much more psychoacoustics. It would take some years for a good team to apply best known models, and then those are incomplete...
ruxvilti'a

Measuring audio quality degradation

Reply #24
What about measuring that digitally?

What if you went through all that trouble  but your ears/personal taste didn't agree with it? Whose last word would prevail in the end? Yours or an algorithm's?
Listen to the music, not the media it's on.
União e reconstrução