Vorbis Listening Test
Reply #16 – 2004-04-05 01:01:17
ff123: Was that analysis done using the tool you describe here ? Yes. That's the only analytical tool for listening test results I know of that is publicly available.It looks perfect for choosing which encoder. So after we receive say 10 results from 10 different people, do we run this on each listener's results or on the whole group? You must first create result tables for each sample. One column for each encoder tested, and one line for each listener. So, you'll end up with 12 tables. Run each one of these tables through Friedman, and you'll get something similar to this:friedman.exe -a results05.txt FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/ Blocked ANOVA analysis Number of listeners: 20 Critical significance: 0.05 Significance of data: 2.70E-002 (significant) --------------------------------------------------------------- ANOVA Table for Randomized Block Designs Using Ratings Source of Degrees Sum of Mean variation of Freedom squares Square F p Total 99 67.80 Testers (blocks) 19 40.26 Codecs eval'd 4 3.65 0.91 2.91 2.70E-002 Error 76 23.89 0.31 --------------------------------------------------------------- Fisher's protected LSD for ANOVA: 0.353 Means: Compaact Real Faac iTunes Nero 4.68 4.37 4.32 4.21 4.11 <==== ---------------------------- p-value Matrix --------------------------- Real Faac iTunes Nero Compaact 0.080 0.049* 0.011* 0.002* Real 0.822 0.400 0.163 Faac 0.537 0.240 iTunes 0.574 ----------------------------------------------------------------------- Compaact is better than Faac, iTunes, Nero (from the latest aac test) Now, notice the line I put a <==== above. Take that same line from each sample result, and create a final table. Run this table through Friedman, and you'll get your final ranking.