IPB

Welcome Guest ( Log In | Register )

Ideas on how to test if video quality affects perceived audio quality?, [was “Scientific method to use (College Dissertation Help!)”]
MikeyTSH
post Jan 14 2013, 00:59
Post #1





Group: Members
Posts: 4
Joined: 14-January 13
Member No.: 105855



Not sure if this is the right forum to post this in, if not, feel free to move the topic and apologies mods!

Anyhoo, I'm in University in England doing my final year dissertation. I want to see if visual stimuli affects the perception of audio quality. The plan is to subject test subjects to WAV, MP3 and AAC files with the latter two encoded at 320, 256, 128 and 96. After ABX testing the MP3 to the AAC at each bit rate, I then want to throw video in at 1080p, 720p, 480p and 240p. Test subjects will do an ABX on the audio but I want to see if a corresponding/conflicting video quality will affect their choices.

An idea I had was the have the audio quality drop throughout the video too but I'm having real trouble working out the best way to test this. I've already made a test "scaling" file, simply putting cross-fades between each file to make the transition as seamless as possible but my big question is really what testing method would I use here? I can hardly say "which do you think sounds better" because the audio is going to degrade in quality. I really want to ask them to hit the pause button when they believe the audio quality has changed but I'm worried about 2 things - 1) will this cause preemptive pressing of the button 2) if I can get around this, what kind of testing method would I use? The idea for this test is to have 2 video files - one remaining at 1080p and then another that degrades in time with the speed that the audio quality degrades, to link the visual stimuli hypothesis. But here again I hit a brick wall - do I ask either a) when did the audio quality change or b) do you think the audio quality changed, in addition to the question "which one sounded better" - my lecturer suggested that having both MP3 and AAC can cement my hypothesis in BSc territory as I can directly hypothesize based on how the codec "technically" work.

Any and all input is greatly appreciated, I look on the internet for some answers, end up going, for example, "oh, null testing, what's that?" and then 20 links later I'm down a rabbit hole that has me scratching my head and, unfortunately, procrastinating.

Thanks again!
Mike

This post has been edited by MikeyTSH: Jan 14 2013, 01:07
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Dynamic
post Jan 14 2013, 23:05
Post #2





Group: Members
Posts: 832
Joined: 17-September 06
Member No.: 35307



First thing is that headphones are more likely to reveal artifacts than speakers. Artifact training may also help

Try some ABX tests yourself. This is best done to compare the lossless original (an experimental control) to an encoded file. ABC/HR is also useful to compare multiple codecs against the lossless control (or various 'anchors')

Well encoded MP3 (e.g. LAME VBR) is remarkably good down to 128 kbps (stereo) and needs careful attention and potentially artifact training often even at 96 kbps (stereo) where it's falling down more dramatically. I guess you'd use CBR, which isn't quite so good.

Decent AAC is likely to be pretty darned good at 96 kbps (stereo), but FAAC for example, is considered a fairly poor encoder, so it depends on the encoder. It might be hard to show a distinct effect without going lower in bitrate. Above about 160 or 192 kbps, both AAC and MP3 are likely to be extremely hard to distinguish from lossless.

I can't see gradual change in quality being viable. Wouldn't it be better to compare pairs of videos together, e.g.

1080p AAC256 vs 720p AAC256
1080p AAC096 vs 720p AAC256
1080p AAC096 vs 720p AAC096
1080p AAC256 vs 720p AAC096

You can then separate out true audio differences from video-induced differences.
Go to the top of the page
+Quote Post

Posts in this topic


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th December 2014 - 15:57