Angoff and Nedelsky Standard Setting Procedures: Implications for the Validity of Proficiency Test Score Interpretation

Abstract

The Angoff and Nedelsky standard setting procedures are currently in use in many educational proficiency testing situations. This study has examined how local content specialists performed when applying these procedures to objective-referenced instruments in reading and mathematics. Research questions addressed several issues related to the validity of test score interpretation, such as the effects of judges' demographic characteristics on ratings, the consistency of cut scores for different groups of judges applying the same method, the cut score differences between the two methods, and the pattern of interrater correlations among item ratings. The results revealed several differences between the standard setting procedures in terms of both level and consistency of the cut scores generated. Also, a possible content-related interaction with the standard setting method was observed.