Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Need listening test grades statistics (Read 8215 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Need listening test grades statistics

Could anybody recommend some articles/papers on listening tests with statistical analysis of collected grades (mean, variance, distribution, normality …) or may be with listings of raw data sets. Going to work out some procedure for automated exclusion of outliers from grades are being received by SoundExpert. As there is no possibility to post-screen “insensitive” listeners in SE tests design the data themselves are the only objects for analysis.
keeping audio clear together - soundexpert.org

Need listening test grades statistics

Reply #1
Maybe a look at the procedures the EBU used in their 2007 multichannel listening test can help you. On page 19 they list three conditions of post-screening.

Chris
If I don't reply to your reply, it means I agree with you.

Need listening test grades statistics

Reply #2
On page 19 they list three conditions of post-screening.

Thank you Chris. Although those conditions are not applicable in SE case directly, the distributions of scores in Appendix 5 are really helpful.
keeping audio clear together - soundexpert.org

Need listening test grades statistics

Reply #3
You might do better to consider partitioning the data, to see if you have sets of listeners who are reacting very differently to the auditory cues.

This way, you can find out if there are multiple listening modalities, which we all know do exist, but nobody really has attempted to sort out the "outliers' who use (for specific example) more emphasis on spatial issues.

This way you can also separate out the random outliers from groups of "outliers" who are in fact using different hearing modalities.
-----
J. D. (jj) Johnston

Need listening test grades statistics

Reply #4
You might do better to consider partitioning the data, to see if you have sets of listeners who are reacting very differently to the auditory cues.

If you mean clusterization of listeners according to their listening (auditory) specificities, then it is interesting. AFAIK in standard listening tests the listeners whose grades are not correlated enough with majority of listeners are just removed. May be in case of large sample of grades there is possibility to pick out different groups of listeners ... In any case this can't be applied to SE grades because they are all anonymous. Only recent grades are stored with IP addresses which can't be used for reliable identification of their authors as well. The problem is to remove only obvious outlying grades. It was done manually but starts to consume more and more time. So some simple and reliable procedure is necessary.
keeping audio clear together - soundexpert.org

Need listening test grades statistics

Reply #5
I've done some research of grades submitted to SE. The purpose was to find simple and adjustable mechanism to reject obvious outlying grades. Finally the mechanism looks as follows:
  • Median ± 1.5 for inlying grades
  • min. number of grades = 15[/b]

I would like to ask people who conducted any listening tests (especially here @HA) and have raw sets of collected grades either to test the above rejection rule or to send those grades to SE. Grades could be depersonalized, just grade sets. Thanks in advance!
keeping audio clear together - soundexpert.org