Differential Item Functioning and Measurement Invariance of Standardized Music Assessments – an Issue of Test-Fairness

Abstract

Test-fairness is one of the most important requirements when results are interpreted with regard to group-differences. Often, group comparisons are made along social categorizations. However, the comparison of groups is context sensitive, i.e. results will be biased as long as only a limited number of context-variables are taken into account. The aim of this paper is twofold: First, we describe and discuss the relevance and theoretical assumptions of test fairness notably in view of an evidence based educational policy. After different aspects of test fairness are highlighted, two major statistical methods (Measurement Invariance Analysis, Differential Item Functioning) are described. In a second step, we use data sets from two studies to investigate test bias for two different music assessments. The samples consist of n = 499 sixth graders (study I) and n = 773 seventh graders (study II) respectively. We define subgroups of the samples in accordance with theoretical assumptions and findings from previous studies about disadvantaged groups in music assessments. We find evidence for test bias with regard to gender, language usage, cognitive ability and experience with music tuition. The results indicate the relevance of test fairness in music assessments, especially, if recommendations for educational practice and policy are made.