An Analysis of Ratings and Interrater Reliability of Adjudicated Orchestra Festivals

Abstract

Abstract The purpose of this study was to identify the influence of school level (middle and high), festival level (district and state), music classification (A, B, C, D, and E), and instrumentation (full and string orchestra) on the overall performance ratings assigned to orchestral performances at large ensemble Music Performance Assessment (MPA) festivals. Data consisted of overall ratings (N = 1,996) assigned at district (n = 1,844) and state level (n = 152) MPA festivals sponsored by the Florida Orchestra Association between 2010 and 2015. Analysis revealed that 94% of all high school orchestras at district and state festivals earned a I/Superior or II/Excellent rating. Logistic regression models revealed: (a) high school orchestras were more likely to earn I/Superior ratings at district festivals than state festivals, (b) middle school orchestras were less likely to earn I/Superior ratings than high school orchestras, and (c) orchestras that performed more difficult repertoire increased their likelihood of earning a I/Superior over ensembles that played easier literature. Festival administrators may use the results of this study to identify specific evaluation trends at adjudicated festivals that can be strengthened.