Welcome Guest ( Log In | Register )

Ideas on how to test if video quality affects perceived audio quality?, [was “Scientific method to use (College Dissertation Help!)”]
post Jan 14 2013, 00:59
Post #1

Group: Members
Posts: 4
Joined: 14-January 13
Member No.: 105855

Not sure if this is the right forum to post this in, if not, feel free to move the topic and apologies mods!

Anyhoo, I'm in University in England doing my final year dissertation. I want to see if visual stimuli affects the perception of audio quality. The plan is to subject test subjects to WAV, MP3 and AAC files with the latter two encoded at 320, 256, 128 and 96. After ABX testing the MP3 to the AAC at each bit rate, I then want to throw video in at 1080p, 720p, 480p and 240p. Test subjects will do an ABX on the audio but I want to see if a corresponding/conflicting video quality will affect their choices.

An idea I had was the have the audio quality drop throughout the video too but I'm having real trouble working out the best way to test this. I've already made a test "scaling" file, simply putting cross-fades between each file to make the transition as seamless as possible but my big question is really what testing method would I use here? I can hardly say "which do you think sounds better" because the audio is going to degrade in quality. I really want to ask them to hit the pause button when they believe the audio quality has changed but I'm worried about 2 things - 1) will this cause preemptive pressing of the button 2) if I can get around this, what kind of testing method would I use? The idea for this test is to have 2 video files - one remaining at 1080p and then another that degrades in time with the speed that the audio quality degrades, to link the visual stimuli hypothesis. But here again I hit a brick wall - do I ask either a) when did the audio quality change or b) do you think the audio quality changed, in addition to the question "which one sounded better" - my lecturer suggested that having both MP3 and AAC can cement my hypothesis in BSc territory as I can directly hypothesize based on how the codec "technically" work.

Any and all input is greatly appreciated, I look on the internet for some answers, end up going, for example, "oh, null testing, what's that?" and then 20 links later I'm down a rabbit hole that has me scratching my head and, unfortunately, procrastinating.

Thanks again!

This post has been edited by MikeyTSH: Jan 14 2013, 01:07
Go to the top of the page
+Quote Post
Start new topic
post Jan 14 2013, 23:05
Post #2

Group: Members
Posts: 835
Joined: 17-September 06
Member No.: 35307

First thing is that headphones are more likely to reveal artifacts than speakers. Artifact training may also help

Try some ABX tests yourself. This is best done to compare the lossless original (an experimental control) to an encoded file. ABC/HR is also useful to compare multiple codecs against the lossless control (or various 'anchors')

Well encoded MP3 (e.g. LAME VBR) is remarkably good down to 128 kbps (stereo) and needs careful attention and potentially artifact training often even at 96 kbps (stereo) where it's falling down more dramatically. I guess you'd use CBR, which isn't quite so good.

Decent AAC is likely to be pretty darned good at 96 kbps (stereo), but FAAC for example, is considered a fairly poor encoder, so it depends on the encoder. It might be hard to show a distinct effect without going lower in bitrate. Above about 160 or 192 kbps, both AAC and MP3 are likely to be extremely hard to distinguish from lossless.

I can't see gradual change in quality being viable. Wouldn't it be better to compare pairs of videos together, e.g.

1080p AAC256 vs 720p AAC256
1080p AAC096 vs 720p AAC256
1080p AAC096 vs 720p AAC096
1080p AAC256 vs 720p AAC096

You can then separate out true audio differences from video-induced differences.
Go to the top of the page
+Quote Post
post Jan 15 2013, 13:34
Post #3

Group: Members
Posts: 4
Joined: 14-January 13
Member No.: 105855

Sorry for the double post but just thought I'd ask - do you know of anywhere I could get a program that does ABX with video support? I have foobar with that extension to allow for ABX.

That said - would I just get them to watch the video but TELL the focus on the audio (I'm wondering whether I should TELL them to listen for which sounds worse / how to phrase the question to them). I want to do an audio only test first to null out any possibly issues. Also, what should I be taking into account for in regards to listening medium? I was thinking of doing it on headphones but what volume would be the best? I want to try and be as level as I can with the fletcher/munson curve...)

This post has been edited by db1989: Jan 15 2013, 14:31
Reason for edit: deleting pointless full quote of post #2
Go to the top of the page
+Quote Post

Posts in this topic

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:


RSS Lo-Fi Version Time is now: 27th November 2015 - 13:30