Welcome Guest ( Log In | Register )

Alternative/Supplement to listening test, Can we compare files sample by sample?
post Feb 16 2004, 17:03
Post #1

Group: Members
Posts: 3
Joined: 16-February 04
Member No.: 12041

I'm new to this forum and I spent a bunch of time reading posts yesterday and today trying to absorb what I could from past discussions. In light of the comments by other newbies about their difficulties hearing artifacts in some of the listening tests, I was thinking maybe there is a way to quantitatively compare files.

This may already have been done and discussed to death. If so, if someone could direct me to such discussions, I would appreciate it.

Otherwise, this is how I imagine such a system working. You start with some reference wave file. You encode that into whatever format you are interested in and then decode it back into a wave file. The comparison program you've written then compares the waveforms of the two files and creates some kind of measure for the deviation of the encoded file from the reference.

The first and simplest measure that comes to mind is simply some kind of mean squared deviation per sample. In this case, you read in the value of both waveforms at each sample and subtract one from the other and square this difference to avoid positive and negative deviations from summing to zero. You sum all these differences over the length of your file and at the end you divide it by the number of samples in the file you used.

At this point, it's not clear to me that this would be a meaningful metric. There may be more meaningful, though more complicated ways to measure the deviation of the encoded file from the reference. The ideal would obviously be if a metric could be defined such that there was a clear correlation between numerical deviation from the reference and listener-perceived deviation from the reference.

I know that ultimately, what we are looking for from a codec is not necessarily complete faithfulness in terms of reproduction of reference waveform, but rather psychoacoustic equivalence: we just want the encoded file to sound the same as the reference. So, this type of analysis is not a replacement for a listening test, but it could be an interesting additional piece of information to have.

I don't feel like I have the expertise or the time to write such a program myself (at least not right now) but please let me know your thoughts on this.
Go to the top of the page
+Quote Post
Start new topic
post Feb 16 2004, 17:11
Post #2

Mad Scientist

Group: Developer (Donating)
Posts: 4901
Joined: 24-September 01
Member No.: 13

To provide a simple summary:

it's not possible (so far) to create an algorithm to determine how good a file sounds compared to the original, and does so reliably and as well as a human listener.

If better algorithms than what we have now were invented, they'd be used in the codecs, and your measure would be useless again.

A simple example is that there exists a PEQUAL (sp?) utility that uses psychoacoustic methods to determine how similar two files sound. But the psychoacoustics used in the encoders are much more advanced than this model, so the measure becomes useless because the codecs are smarter than the evaluation software.

I'm sure this has been discussed here before, too.
Go to the top of the page
+Quote Post

Posts in this topic

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:


RSS Lo-Fi Version Time is now: 30th November 2015 - 19:16