An Examination of Differential Item Functioning in a Rubric to Assess Solo Music Performance

Abstract

The purpose of this study was to examine differential item functioning (DIF) in a rubric used to assess middle-school solo and ensemble performances. This study was guided by the following research questions: (a) does measurement equivalence for all items exist when used to measure subgroups of students based on their musical instrument? (b) what patterns of differential item functioning effects exist for items when used to measure subgroups of students based on their musical instrument? and (c) what size of differential item functioning effects exists for items when used to measure subgroups of students based on their musical instrument? In total, 17 adjudicators evaluated 138 middle-school instrumental students (ages 11–13) in the context of a live, formal solo and ensemble performance assessment. Using the Many Facets Rasch Partial Credit measurement model, measurement equivalence for all items did not exist when used to measure subgroups of students based on their musical instrument ([Formula: see text]2 (252) = 634.00, p < .01). Of the 252 total pairwise interactions examined between items and instruments, 57 (22.62%) significant interaction terms were identified. Overall, 26 (10.3%) significant interaction terms demonstrated a moderate to large effect (|DIF| >/= .63 logits), nine (3.6%) significant interaction terms demonstrated a slight to moderate effect (|DIF| = (.43, .63 logits)), and 22 (8.7%) significant interaction terms demonstrated a negligible effect (|DIF| < .43 logits). Implications for the fairness of music performance assessments and improved assessment protocols are discussed.