Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Alternative/Supplement to listening test (Read 17838 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Alternative/Supplement to listening test

I'm new to this forum and I spent a bunch of time reading posts yesterday and today trying to absorb what I could from past discussions.  In light of the comments by other newbies about their difficulties hearing artifacts in some of the listening tests, I was thinking maybe there is a way to quantitatively compare files. 

This may already have been done and discussed to death.  If so, if someone could direct me to such discussions, I would appreciate it. 

Otherwise, this is how I imagine such a system working.  You start with some reference wave file.  You encode that into whatever format you are interested in and then decode it back into a wave file.  The comparison program you've written then compares the waveforms of the two files and creates some kind of measure for the deviation of the encoded file from the reference. 

The first and simplest measure that comes to mind is simply some kind of mean squared deviation per sample.  In this case, you read in the value of both waveforms at each sample and subtract one from the other and square this difference to avoid positive and negative deviations from summing to zero.  You sum all these differences over the length of your file and at the end you divide it by the number of samples in the file you used. 

At this point, it's not clear to me that this would be a meaningful metric.  There may be more meaningful, though more complicated ways to measure the deviation of the encoded file from the reference.  The ideal would obviously be if a metric could be defined such that there was a clear correlation between numerical deviation from the reference and listener-perceived deviation from the reference.

I know that ultimately, what we are looking for from a codec is not necessarily complete faithfulness in terms of reproduction of reference waveform, but rather psychoacoustic equivalence: we just want the encoded file to sound the same as the reference.  So, this type of analysis is not a replacement for a listening test, but it could be an interesting additional piece of information to have.

I don't feel like I have the expertise or the time to write such a program myself (at least not right now) but please let me know your thoughts on this.

 

Alternative/Supplement to listening test

Reply #1
To provide a simple summary:

it's not possible (so far) to create an algorithm to determine how good a file sounds compared to the original, and does so reliably and as well as a human listener.

If better algorithms than what we have now were invented, they'd be used in the codecs, and your measure would be useless again.

A simple example is that there exists a PEQUAL (sp?) utility that uses psychoacoustic methods to determine how similar two files sound. But the psychoacoustics used in the encoders are much more advanced than this model, so the measure becomes useless because the codecs are smarter than the evaluation software.

I'm sure this has been discussed here before, too.


Alternative/Supplement to listening test

Reply #3
Yes, this question is in the MP3 FAQ. However, Garf's answer being very synthetic and clear, I'm replacing the old FAQ link with this one.
You can still read more explanations here : http://www.hydrogenaudio.org/forums/index....t=ST&f=1&t=5838

Alternative/Supplement to listening test

Reply #4
And this discussion might be what you're looking for.
The object of mankind lies in its highest individuals.
One must have chaos in oneself to be able to give birth to a dancing star.

Alternative/Supplement to listening test

Reply #5
Thanks for the links to previous discussions.  I'm glad that other people have thought about this stuff before and that I'm not way off base for thinking this way.  I actually mentioned to a friend yesterday that I would be interested in hearing the difference between a wave and what results after encoding and decoding.  Maybe I'll do this for fun anyway even though it's been done before.