IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
Measuring audio quality degradation
birdie
post Feb 13 2014, 13:09
Post #1





Group: Members
Posts: 108
Joined: 3-March 06
From: this planet
Member No.: 28235



Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

My question is how one can determine the "best" track out of many tracks having different bitrates or/and codecs?

For instance 192Kbit MP3 track produced using a CDDA source is definitely better than another one which is a reencode from 128Kbit MP3 to 320Kbit MP3 just for the sake of "Yo-ho, we have 320KBit tracks here".

Another possible source of audios are reencodes from YouTube lousy audio into yet another codec (usually 320KBit mp3).

Is there a way to compare different musical files and say conclusively that this or that one has the highest quality? I.e. the least amount of distortions and the widest dynamic range.

Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Feb 13 2014, 13:55
Post #2





Group: Members
Posts: 3689
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (birdie @ Feb 13 2014, 07:09) *
Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

My question is how one can determine the "best" track out of many tracks having different bitrates or/and codecs?

For instance 192Kbit MP3 track produced using a CDDA source is definitely better than another one which is a reencode from 128Kbit MP3 to 320Kbit MP3 just for the sake of "Yo-ho, we have 320KBit tracks here".

Another possible source of audios are reencodes from YouTube lousy audio into yet another codec (usually 320KBit mp3).

Is there a way to compare different musical files and say conclusively that this or that one has the highest quality? I.e. the least amount of distortions and the widest dynamic range.



(1) Become a trained listener

(2) Exercise that gift.
Go to the top of the page
+Quote Post
birdie
post Feb 13 2014, 14:23
Post #3





Group: Members
Posts: 108
Joined: 3-March 06
From: this planet
Member No.: 28235



What about measuring that digitally?

Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.
Go to the top of the page
+Quote Post
julf
post Feb 13 2014, 14:24
Post #4





Group: Members
Posts: 223
Joined: 14-January 12
Member No.: 96426



QUOTE (birdie @ Feb 13 2014, 14:23) *
Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.


Most people can't. And if you can't tell a difference by listening, why would you care?
Go to the top of the page
+Quote Post
birdie
post Feb 13 2014, 14:53
Post #5





Group: Members
Posts: 108
Joined: 3-March 06
From: this planet
Member No.: 28235



Because I need to tell which one of two low quality encodes is "better". ;-)

This post has been edited by birdie: Feb 13 2014, 14:54
Go to the top of the page
+Quote Post
dhromed
post Feb 13 2014, 15:14
Post #6





Group: Members
Posts: 1301
Joined: 16-February 08
From: NL
Member No.: 51347



What is "better"?
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Feb 13 2014, 15:30
Post #7





Group: Members
Posts: 3689
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (birdie @ Feb 13 2014, 08:23) *
What about measuring that digitally?


I lecture people all day about how well technical tests inform us about sound quality, but to this day there does not appear to be a generally accepted 100% reliable way to do that with perceptually coded files.

Oh, technical tests can help to a degree - for example if a 44/16 file has an obvious brick wall filter @ 15-18 KHz we probably can guess how it got there and there's a perceptual coder in the story.

QUOTE
Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.


Most people can't ever do it, and even those who sometimes can, most of that lucky group can't do it 100% of the time.

To some degree this is like a discussion as to whether a certain person is using certain err, clothes items to augment their appearance... ;-)
Go to the top of the page
+Quote Post
greynol
post Feb 13 2014, 17:32
Post #8





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (birdie @ Feb 13 2014, 04:09) *
Is there a way to compare different musical files and say conclusively that this or that one has the highest quality?

Yes, a double blind AB test, perhaps a forced choice test.

QUOTE
I.e. the least amount of distortions and the widest dynamic range.

No. Are you familiar with any mp3 encoder that compromises dynamic range at the bitrates you mentioned? I'm not; and I doubt such an encoder exists. "Distortions" is too vague, but assuming you're talking about audible artifacts, you're asking for an algorithm that knows something about psychoacoustics. Read the first part of my post again to see what is a proper way to judge on that metric.

This post has been edited by greynol: Feb 13 2014, 17:34


--------------------
Concern trolls: not a myth.
Go to the top of the page
+Quote Post
2Bdecided
post Feb 13 2014, 18:11
Post #9


ReplayGain developer


Group: Developer
Posts: 5089
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (Arnold B. Krueger @ Feb 13 2014, 14:30) *
I lecture people all day about how well technical tests inform us about sound quality, but to this day there does not appear to be a generally accepted 100% reliable way to do that with perceptually coded files.
...and most of the methods that do exist require access to both the original (perfect, lossless) version as well as the messed up (lossy) version, to estimate how bad the messed up version sounds. They can't compare two messed up versions and tell you which one is more faithful to an unknown original, or which one sounds "better".

Cheers,
David.

This post has been edited by 2Bdecided: Feb 13 2014, 18:11
Go to the top of the page
+Quote Post
greynol
post Feb 13 2014, 18:32
Post #10





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Better does not necessarily mean most faithful to the reference. In order to have a preference you don't need access to the reference at all.

Trivial case in point: .5 dB amplification vs 1 dB amplification when neither results in audible distortion.

What if the listener likes the sound of artifacts specific to a particular encoding?


--------------------
Concern trolls: not a myth.
Go to the top of the page
+Quote Post
KozmoNaut
post Feb 13 2014, 18:39
Post #11





Group: Members
Posts: 85
Joined: 25-January 07
Member No.: 39990



QUOTE (julf @ Feb 13 2014, 14:24) *
QUOTE (birdie @ Feb 13 2014, 14:23) *
Besides with my audio equipment I cannot even tell the difference between good 192Kbit MP3 encodes and CDDA audio.


Most people can't. And if you can't tell a difference by listening, why would you care?


Exactly. The best version of a given track is the one you like the most, not the one that's measures a certain way that is determined to be "best".

Subjective listening tests are often (rightfully) ridiculed, but for finding which track you like best, only your own subjective choice matters.
Go to the top of the page
+Quote Post
2Bdecided
post Feb 13 2014, 19:29
Post #12


ReplayGain developer


Group: Developer
Posts: 5089
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (greynol @ Feb 13 2014, 17:32) *
Better does not necessarily mean most faithful to the reference.
I know, that's why I put "or". And I put better in quotes.

I was just saying: the automated methods that do exist can't answer the OP's question, even if you're willing to overlook their inferiority to human ears in the tasks they are designed for.

Some people did look at blind quality assessment (i.e. you don't have the original, but how bad is the version that you do have); I think they got somewhere with telephone quality speech, but I don't know if they got anywhere with music.

Cheers,
David.
Go to the top of the page
+Quote Post
greynol
post Feb 13 2014, 19:32
Post #13





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



...then let's try it again.

QUOTE (2Bdecided @ Feb 13 2014, 09:11) *
They can't compare two messed up versions and tell you [...] which one sounds "better".

Yes, they most certainly can; with or without the use of quotes.

This post has been edited by greynol: Feb 13 2014, 19:33


--------------------
Concern trolls: not a myth.
Go to the top of the page
+Quote Post
birdie
post Feb 14 2014, 09:32
Post #14





Group: Members
Posts: 108
Joined: 3-March 06
From: this planet
Member No.: 28235



QUOTE (2Bdecided @ Feb 13 2014, 23:11) *
...and most of the methods that do exist require access to both the original (perfect, lossless) version as well as the messed up (lossy) version, to estimate how bad the messed up version sounds. They can't compare two messed up versions and tell you which one is more faithful to an unknown original, or which one sounds "better".

Cheers,
David.


Exactly what I'm talking about.

Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable or appropriate but it seems like its my only choice.

BTW, I'm curious - auCDtect shows such characteristics as "Coefficient of nonlinearity of a phase", "First order smothness" and "Second order smothness" but so far I haven't been able to identify a pattern in these highly cryptic numbers. Does anyone know what they mean and how they can be applied in this situation?

This post has been edited by birdie: Feb 14 2014, 09:41
Go to the top of the page
+Quote Post
Kohlrabi
post Feb 14 2014, 10:00
Post #15





Group: Super Moderator
Posts: 1017
Joined: 12-March 05
From: Kiel, Germany
Member No.: 20561



QUOTE (birdie @ Feb 14 2014, 09:32) *
Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable or appropriate but it seems like its my only choice.
To add to the problem of choosing a version, multiple versions of the same track don't necessarily have to be based on the same source material, or master. It's easy and common to mis-attribute mastering differences to audio codecs (and settings), if one doesn't realize this issue. So two versions of the same song might not only differ in encoder settings and choice, but also in the source material. In such a case, there is really no other meaningful way to choose the song you like best other than trusting your senses, even if it turns out it's the WMA version with the lowest bitrate. cool.gif

This post has been edited by Kohlrabi: Feb 14 2014, 10:21


--------------------
PRaT is the new jitter.
Go to the top of the page
+Quote Post
Porcus
post Feb 14 2014, 10:40
Post #16





Group: Members
Posts: 1842
Joined: 30-November 06
Member No.: 38207



Of course it should be possible to develop an algorithm that ranks signals in an order which highly rank-correlates with subjective quality. That is precisely what the psy-models of lossy encoders do - however they do, naturally, assume access to the lossless reference. Measuring artifacts should also be possible, and some are available: click/pop detection used for vinyl rips, for example, depend upon detecting "something bad" with certain reliability. And someone on this forum pointed out to me (when I asked a releated question) that in video processing, there are artifact removal filters - of course with side effects yes, but if they succeed in presenting a subjectively "better" output, then they are beyond the level differentiating by quality.

Building an algorithm to rank signals, would likely take the same job as the development of psy models: listen, model, and tune (i.e. estimate) the model to mimic human preferences. It shouldn't be hard - assuming you can be satisfied with less than perfection - but it could be very time consuming to do the job.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
2Bdecided
post Feb 14 2014, 12:22
Post #17


ReplayGain developer


Group: Developer
Posts: 5089
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (greynol @ Feb 13 2014, 18:32) *
...then let's try it again.

QUOTE (2Bdecided @ Feb 13 2014, 09:11) *
They can't compare two messed up versions and tell you [...] which one sounds "better".

Yes, they most certainly can; with or without the use of quotes.
Ah, I see what you mean.

Arnie was talking about "technical tests" in the original quote, not human ears. I was talking about the better ear-like technical tests (PEAQ etc).
Go to the top of the page
+Quote Post
2Bdecided
post Feb 14 2014, 12:40
Post #18


ReplayGain developer


Group: Developer
Posts: 5089
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (birdie @ Feb 14 2014, 08:32) *
Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable
Why not? The lossy compression used on YouTube and most other places is designed to destroy the signal as much as possible while fooling human ears that almost nothing has been changed. It's not trying to please an oscilloscope or a spectrogram or a computer - it's trying to please your ears.

Maintaining the measurable signal integrity and quality at a very high level would also please your ears - but it would not let YouTube squeeze something that sounds half-decent into 128kbps or less.

A high quality mp3 has about 20-30dB Signal to Noise Ratio (SNR) - i.e. the added noise is always about 25dB below the music. A cassette tape without Dolby has about 45dB SNR - i.e. the noise from the cassette tape is measurably lower - it measures much better than the mp3. Yet the hiss on the tape is easily audible, while the noise in the mp3 is mostly inaudible. That's because the noise in the mp3 is hidden in areas your ears are insensitive to (it is masked by the music), while the noise on the tape isn't.

There are similar differences between different encoders and formats. Some add more noise (so measure worse) but hide it better (as far as human ears are concerned) so sound better.

The absolute best automatic measurements of lossy audio encoding quality try to mimic human ears - but they can't do it perfectly. That's why the gold standard for assessing the quality of audio codecs is a carefully controlled listening test using human listeners. (Ditto video codecs and viewing tests).

Cheers,
David.
Go to the top of the page
+Quote Post
Ethan Winer
post Feb 14 2014, 22:32
Post #19





Group: Members
Posts: 248
Joined: 12-May 09
From: New Milford, CT
Member No.: 69730



QUOTE (birdie @ Feb 13 2014, 07:09) *
Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

In my experience with illegally downloaded music, the bigger quality factor is how badly someone altered the music believing they're making it better. I've heard music that was severely EQ'd and overly compressed. So for me, I'd rather have a 128 kbps version of a file taken off a CD, than a 320 kbps version that someone messed with (or recorded off vinyl).

--Ethan


--------------------
I believe in Truth, Justice, and the Scientific Method
Go to the top of the page
+Quote Post
Glenn Gundlach
post Feb 15 2014, 09:34
Post #20





Group: Members
Posts: 366
Joined: 19-April 08
From: LA
Member No.: 52914



It sounds like you want a computer program to tell you which is 'better'. This is better than you actually listening?

When I find problems with my audio system I get out the test equipment to try to find and correct the problem which was originally discovered with my ears. Trust me that your ears are more sensitive than the best test gear but the equipment is getting better.

Go to the top of the page
+Quote Post
birdie
post Feb 15 2014, 16:12
Post #21





Group: Members
Posts: 108
Joined: 3-March 06
From: this planet
Member No.: 28235



QUOTE (Ethan Winer @ Feb 15 2014, 03:32) *
QUOTE (birdie @ Feb 13 2014, 07:09) *
Sometimes it's hard to find the music you're interested in either in digital or in physical form, i.e. you cannot legally buy it anywhere yet certain compositions are readily available on the Internet.

In my experience with illegally downloaded music, the bigger quality factor is how badly someone altered the music believing they're making it better. I've heard music that was severely EQ'd and overly compressed. So for me, I'd rather have a 128 kbps version of a file taken off a CD, than a 320 kbps version that someone messed with (or recorded off vinyl).

--Ethan


Oh, yeah, that's been my experience as well - 320Kb mp3s that sound a lot worse than 128Kbit due to normalization or/and compression.
Go to the top of the page
+Quote Post
Porcus
post Feb 16 2014, 12:52
Post #22





Group: Members
Posts: 1842
Joined: 30-November 06
Member No.: 38207



QUOTE (2Bdecided @ Feb 14 2014, 12:40) *
QUOTE (birdie @ Feb 14 2014, 08:32) *
Seems like the sentiment here is to trust my ears. That's not exactly what I find acceptable
Why not?


QUOTE (Glenn Gundlach @ Feb 15 2014, 09:34) *
It sounds like you want a computer program to tell you which is 'better'. This is better than you actually listening?




Well first of all a good algorithm saves human labour. If you have a million files to test, then I would argue that an algorithm with a 0.75 rank correlation to your ears, is a better practical solution than listening to all of them.

Second, a statistical model may be way more consistent than human judgement. Obviously if you are judging for others; four ears hear better than two, and a model built on many ears' experience could be way superior to my pair of ears if the purpose is to be a proxy for your pair of ears. (There is even nothing that says it isn't a superior model of your ears as of tomorrow either, if you are having a bad ear day.)

Third, for lossy encoding and artifact removal - e.g. pops/clicks - the algorithm must know what is "better". Your ears can tell that "this second without that click" is an improvement, and that "this lossy file is good enough", but in order to actually get that, you need an algorithm that knows what to replace that split second of signal with, resp. an algorithm with a decent psy model. Otherwise you have to either do sound engineering manually, or let the software generate an enormous lot of more or less random suggestions ... back to the human labour argument, then.



So yes, I would very much like to have computer program to tell me what is better. And luckily there are such ones around for certain purposes.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
Kees de Visser
post Feb 16 2014, 18:15
Post #23





Group: Members
Posts: 658
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (Porcus @ Feb 16 2014, 12:52) *
If you have a million files to test, then I would argue that an algorithm with a 0.75 rank correlation to your ears, is a better practical solution than listening to all of them.
Humans tend to listen in real time at best, so with strict deadlines and huge amounts of data it's more practical to use software to control audio quality. One of the professional applications used by major content providers is Interra's Baton. It's a file based QC system. Advantage over human listening is speed and consistency. When I inquired a few years ago the price tag was about $30k and afaik there isn't a cheaper version available. It can do audio and video QC. The audio part features:
QUOTE
• Silence • Clipping • Mute • Test tones • Phase detection • Wow & Flutter • Audio distortion • Jitter • Transient noise • High Frequency noise • Background noise • Minimum Audio Level • Audio Dropout • Over Modulation Noise • Click and pop • Loudness compliance (ITU, EBU, CALM Act)
I can imagine that for media exchange between content providers this can be a great system. At least you're not depending on a good or bad day of an audio engineer who has to listen to (often) boring material smile.gif
@Glenn: don't you work in a pro A/V environment ? Do you use automated QC ?
Go to the top of the page
+Quote Post
AstralStorm
post Feb 25 2014, 00:58
Post #24





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



Currently the best available tool known to me is PEAQ and it's quite terrible. Does not have enough knowledge of human psychoacoustics...
It's not even 75% accurate for lightly psychoacoustically lossy compressed audio, such as Wavpack lossy mode, much less for more advanced lossy codecs.
It will underrate frequency response errors and channel balance as well, making it quite not useful for QC on, say, headphones. It cannot deal with reverberation (most of the time) and many nonlinearities adequately either.

Now then, feel free to improve it with much more psychoacoustics. It would take some years for a good team to apply best known models, and then those are incomplete...


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
includemeout
post Feb 25 2014, 01:19
Post #25





Group: Members
Posts: 282
Joined: 16-December 09
From: Maringα, Brazil
Member No.: 76067



QUOTE (birdie @ Feb 13 2014, 10:23) *
What about measuring that digitally?

What if you went through all that trouble but your ears/personal taste didn't agree with it? Whose last word would prevail in the end? Yours or an algorithm's?


--------------------
Listen to the music, not the media.
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd August 2014 - 06:00