IPB

Welcome Guest ( Log In | Register )

I think I've discovered a new listening test method, Please help me validate
Jax184
post Mar 3 2011, 23:32
Post #1





Group: Members
Posts: 14
Joined: 3-March 11
From: Vancouver, BC
Member No.: 88675



This is my first post, and I have no professional background in audio, but I love music and try hard to treat it with respect. So I offer you something I've been working on.

I've always had a problem with traditional A/B or ABX listening tests. They're great when the differences are pretty big, but as they get smaller it becomes harder and harder to remember precisely what one track sounded like when you're listening to another. I for one just can't think about audio with the kind of detail that I can with visual information. Now I will confess that by the time this becomes a problem, the quality difference is probably not very significant. But perhaps I'll get better equipment later and suddenly the difference will be more obvious. Or perhaps I'll play the file to someone with better ears and they'll find it unpleasant. Whatever the case, I wanted a way to detect even small differences in audio quality. My concern was not so much with being blind or even with determining which audio file sounded better, just to determine if there was a discernible difference in the first place. In my mind there's no point trying to say which file sounds better if they've just been proven to sound exactly the same. But this method could easily be performed under double blind conditions.

Anyway, I think I've come up with a solution. So far it's worked nicely in my tests, but it seems far too easy. I can't help but suspect I'm overlooking something. So I'm asking for your help to validate the technique. Tell me if you can see a flaw in the process.

My inspiration came from the classic method of inverting one stereo channel and combining it with the other to eliminate center channel vocals and allow analysis of the subtler instrumental stuff. I've heard it called OOPS or Out Of Phase Stereo. I always liked the elegance of it. But then I realized I could do something similar to analyze the impact of various levels of audio compression.

To start with I ripped a single track from a CD as a wav file, then created several versions of it compressed with the latest LAME library at different bitrates. VBR 0, CBR 320, CBR 128 and finally CBR 56 to provide a really glaring example if need be.
Then I loaded the wav into Audacity along with one of the compressed versions, carefully measured and cut off the padding created by the Mp3 compression so the two waveforms were aligned not simply to the second but to the very sample, inverted the compressed version and mixed it with the wav. This produced a file which contained only the portions of the audio which had been removed or altered by LAME during compression. It was really cool to listen to, a hissy scratchy sort of racket. But it's not the difference file itself I was interested in.

I then loaded the corresponding Mp3 back into audacity below this "difference" file. With the two in perfect waveform alignment, playing them together produced something which was mathematically identical to the source CD. But at any time during the playback I could mute the "difference" file and switch instantly to hearing just the Mp3. If there was a discernible quality difference between the Mp3 and the source CD, it would show up at the point of muting as a change in volume, background hiss, clarity, etc. But if the aspects of the file which had been removed or altered were imperceptible to my ears (and thus having no impact on audio quality,) there would be no change when I muted/unmuted the "difference" track.

So far my testing has revealed absolutely no detectable change on the half dozen songs I've tried at CBR 320 or VBR -0. To prove that the method's working though I also made 128 bitrate and 56 bitrate Mp3s. Those produce a dramatic drop in quality every time the "difference" track is muted, but sound just like the CD when it's unmuted. So it seems the method works correctly, and LAME works really damn good these days.

I've also found that it's much easier to detect an increase in quality from unmuting the "difference" track than a decrease in quality from muting it.

So what do you guys think? Has anyone else tried this before? Am I on to something here, or have I misunderstood something? I do hope I'm correct in this because the method is so simple and elegant. It only takes a minute to set up.
Go to the top of the page
+Quote Post

Posts in this topic
- Jax184   I think I've discovered a new listening test method   Mar 3 2011, 23:32
- - greynol   It's a sighted test, so you cannot guarantee t...   Mar 4 2011, 00:02
- - Jax184   Is it actually able to switch between a compressed...   Mar 4 2011, 00:07
- - greynol   Try it for yourself.   Mar 4 2011, 00:09
- - Jax184   For a moment I was quite excited, but every time I...   Mar 4 2011, 00:19
|- - Garf   QUOTE (Jax184 @ Mar 4 2011, 00:19) For a ...   Mar 4 2011, 18:54
- - Jax184   Perhaps I can clarify my intent with this a little...   Mar 4 2011, 01:34
- - DVDdoug   I'll try to take a few minutes later to conver...   Mar 4 2011, 01:42
- - Jax184   Yes, it is a roundabout way to do an a/b, but the ...   Mar 4 2011, 01:52
- - greynol   If you cannot remember what something sounds like,...   Mar 4 2011, 01:56
- - FumbleFinger   Interesting. I've no professional background i...   Mar 4 2011, 04:12
|- - greynol   QUOTE (FumbleFinger @ Mar 3 2011, 19:12) ...   Mar 4 2011, 04:26
||- - FumbleFinger   QUOTE (greynol @ Mar 4 2011, 03:26) QUOTE...   Mar 4 2011, 15:06
|- - saratoga   QUOTE (FumbleFinger @ Mar 3 2011, 22:12) ...   Mar 4 2011, 04:30
|- - Polouess   If the glitches and jitter are the OPs main concer...   Mar 4 2011, 10:00
- - Jax184   I am painfully aware of how fallible our senses ar...   Mar 4 2011, 13:04
- - 2Bdecided   It doesn't really matter whether you're sw...   Mar 4 2011, 15:19
- - Jax184   Ahh, but that's what I'm trying to build...   Mar 4 2011, 15:28
- - Jax184   Here, how about I include a practical example to m...   Mar 4 2011, 16:00
- - 2Bdecided   I agree it would be nice to have an ABX tool with ...   Mar 4 2011, 16:17
|- - NullC   QUOTE (2Bdecided @ Mar 4 2011, 07:17) I a...   Mar 7 2011, 16:53
|- - googlebot   Squishyball implements "glitchlessness" ...   Mar 7 2011, 17:15
|- - NullC   QUOTE (googlebot @ Mar 7 2011, 08:15) Squ...   Mar 7 2011, 21:38
|- - googlebot   Oh, ok. I only knew -B and -M.   Mar 7 2011, 21:53
- - greynol   QUOTE (Jax184 @ Mar 4 2011, 04:04) I...   Mar 4 2011, 18:10
- - SamDeRe81   Alright listen, this guy http://www.youtube.com/wa...   Mar 4 2011, 19:36
- - greynol   I don't see an error signal being mixed in and...   Mar 4 2011, 19:57
- - Jax184   QUOTE (Garf @ Mar 4 2011, 09:54) QUOTE (J...   Mar 4 2011, 20:15
- - DonP   I wonder if the smoothness of Audacity is really t...   Mar 4 2011, 20:25
|- - Jax184   QUOTE (DonP @ Mar 4 2011, 11:25) I wonder...   Mar 4 2011, 20:39
|- - dhromed   QUOTE (Jax184 @ Mar 4 2011, 20:39) Just r...   Mar 4 2011, 22:38
|- - Garf   QUOTE (Jax184 @ Mar 4 2011, 20:39) That...   Mar 5 2011, 10:05
- - AndyH-ha   It isnít so much the senses per se but expectation...   Mar 4 2011, 20:47
- - Jax184   I actually used the wine example in the article I ...   Mar 4 2011, 20:53
|- - FumbleFinger   QUOTE (Jax184 @ Mar 4 2011, 19:53) ...so ...   Mar 4 2011, 21:01
- - greynol   If you chose a part of your adulterated sample to ...   Mar 4 2011, 21:41
- - Jax184   This method would be pointless for comparing the s...   Mar 4 2011, 21:43
|- - greynol   QUOTE (Jax184 @ Mar 4 2011, 12:43) It...   Mar 4 2011, 21:47
- - Jax184   QUOTE (greynol @ Mar 4 2011, 12:47) QUOTE...   Mar 4 2011, 22:11
- - greynol   I think you need to demonstrate objectively that t...   Mar 4 2011, 22:28
|- - FumbleFinger   QUOTE (greynol @ Mar 4 2011, 21:28) peopl...   Mar 5 2011, 03:49
|- - greynol   QUOTE (FumbleFinger @ Mar 4 2011, 18:49) ...   Mar 5 2011, 04:16
- - Jax184   This is absurd. 1. The test can be performed dou...   Mar 5 2011, 04:58
|- - Soap   QUOTE (Jax184 @ Mar 4 2011, 22:58) You kn...   Mar 5 2011, 05:08
|- - sthayashi   QUOTE (Jax184 @ Mar 4 2011, 22:58) This i...   Mar 5 2011, 07:51
|- - FumbleFinger   QUOTE (Jax184 @ Mar 5 2011, 03:58) This i...   Mar 5 2011, 16:12
|- - dhromed   QUOTE (FumbleFinger @ Mar 5 2011, 16:12) ...   Mar 5 2011, 16:47
||- - gaekwad2   QUOTE (dhromed @ Mar 5 2011, 17:47) QUOTE...   Mar 5 2011, 17:14
||- - greynol   QUOTE (gaekwad2 @ Mar 5 2011, 08:14) I th...   Mar 5 2011, 19:23
|- - greynol   QUOTE (FumbleFinger @ Mar 5 2011, 07:12) ...   Mar 5 2011, 19:32
|- - FumbleFinger   QUOTE (greynol @ Mar 5 2011, 18:32) ...If...   Mar 7 2011, 01:54
- - greynol   You're trying to reinvent the wheel when there...   Mar 5 2011, 05:08
- - greynol   You also need buttons to make sure playback is beg...   Mar 5 2011, 08:54
|- - 2Bdecided   QUOTE (greynol @ Mar 5 2011, 07:54) The m...   Mar 7 2011, 10:28
|- - greynol   QUOTE (2Bdecided @ Mar 7 2011, 01:28) If ...   Mar 7 2011, 18:27
|- - 2Bdecided   QUOTE (greynol @ Mar 7 2011, 17:27) QUOTE...   Mar 7 2011, 20:31
- - Garf   By the way, the ABX program included with LAME (I...   Mar 5 2011, 10:13
- - greynol   Simple example: -0.6 requires summing 1.2 in order...   Mar 7 2011, 20:35


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th July 2014 - 09:34