Judgmental Standard Setting: The Development of Objective Content and Performance Standards for Secondary-Level Solo Instrumental Music Assessment

Abstract

The purpose of this study was to describe the development of content and performance standards for a rubric to evaluate secondary-level solo instrumental music performance using a modified bookmark standard setting procedure. The research questions that guided this study include (1) What are the psychometric qualities of a rubric to evaluate secondary-level solo instrumental music performance? (2) What is the quality of ratings obtained for the standard-setting panel of subject matter expert judges? (3) What cut scores best categorize secondary-level solo instrumental performances into four performance levels across the latent performance achievement variable? and (4) What content mastery of items best categorizes achievement in secondary-level solo music performance at each of the four performance levels? A panel of eight subject matter experts participated in the study. A 30-item rubric was used to collect the judging panel’s observed responses. The collected responses were transformed to linear measures using the multifaceted Rasch partial credit model. The bookmark procedure resulted in the setting of three cut points representing minimum pass levels on a latent continuum differentiating between four performance achievement levels (rudimentary, emerging, proficient, and exemplary) with clearly defined content standards. Implications for opportunity to learn are discussed.