IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Need listening test grades statistics, … or raw data sets
Serge Smirnoff
post Jan 10 2010, 13:05
Post #1





Group: Members
Posts: 371
Joined: 14-December 01
Member No.: 641



Could anybody recommend some articles/papers on listening tests with statistical analysis of collected grades (mean, variance, distribution, normality …) or may be with listings of raw data sets. Going to work out some procedure for automated exclusion of outliers from grades are being received by SoundExpert. As there is no possibility to post-screen “insensitive” listeners in SE tests design the data themselves are the only objects for analysis.


--------------------
keeping audio clear together - soundexpert.org
Go to the top of the page
+Quote Post
C.R.Helmrich
post Jan 10 2010, 15:51
Post #2





Group: Developer
Posts: 690
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



Maybe a look at the procedures the EBU used in their 2007 multichannel listening test can help you. On page 19 they list three conditions of post-screening.

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
Serge Smirnoff
post Jan 10 2010, 18:00
Post #3





Group: Members
Posts: 371
Joined: 14-December 01
Member No.: 641



QUOTE (C.R.Helmrich @ Jan 10 2010, 17:51) *
On page 19 they list three conditions of post-screening.

Thank you Chris. Although those conditions are not applicable in SE case directly, the distributions of scores in Appendix 5 are really helpful.


--------------------
keeping audio clear together - soundexpert.org
Go to the top of the page
+Quote Post
Woodinville
post Jan 10 2010, 21:51
Post #4





Group: Members
Posts: 1402
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



You might do better to consider partitioning the data, to see if you have sets of listeners who are reacting very differently to the auditory cues.

This way, you can find out if there are multiple listening modalities, which we all know do exist, but nobody really has attempted to sort out the "outliers' who use (for specific example) more emphasis on spatial issues.

This way you can also separate out the random outliers from groups of "outliers" who are in fact using different hearing modalities.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Serge Smirnoff
post Jan 10 2010, 23:47
Post #5





Group: Members
Posts: 371
Joined: 14-December 01
Member No.: 641



QUOTE (Woodinville @ Jan 10 2010, 23:51) *
You might do better to consider partitioning the data, to see if you have sets of listeners who are reacting very differently to the auditory cues.

If you mean clusterization of listeners according to their listening (auditory) specificities, then it is interesting. AFAIK in standard listening tests the listeners whose grades are not correlated enough with majority of listeners are just removed. May be in case of large sample of grades there is possibility to pick out different groups of listeners ... In any case this can't be applied to SE grades because they are all anonymous. Only recent grades are stored with IP addresses which can't be used for reliable identification of their authors as well. The problem is to remove only obvious outlying grades. It was done manually but starts to consume more and more time. So some simple and reliable procedure is necessary.


--------------------
keeping audio clear together - soundexpert.org
Go to the top of the page
+Quote Post
Serge Smirnoff
post Feb 16 2011, 20:07
Post #6





Group: Members
Posts: 371
Joined: 14-December 01
Member No.: 641



I've done some research of grades submitted to SE. The purpose was to find simple and adjustable mechanism to reject obvious outlying grades. Finally the mechanism looks as follows:
  • Median ± 1.5 for inlying grades
  • min. number of grades = 15

I would like to ask people who conducted any listening tests (especially here @HA) and have raw sets of collected grades either to test the above rejection rule or to send those grades to SE. Grades could be depersonalized, just grade sets. Thanks in advance!


--------------------
keeping audio clear together - soundexpert.org
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th October 2014 - 19:59