A Model for Setting Performance Standards for Standardized Patient Examinations

Abstract

Setting standards for complex performance assessments is difficult. This report describes and evaluates a model for setting performance standards for standardized patient examinations. A variation of a modified Angoff standard-setting procedure was used to set cutoffs for a standardized patient exercise. Analysis of the procedure was based on evaluation of the impact of the iterative process, the ability of the final standards to accurately classify trainees and the judges’confidence in the procedure and final standards. The practicality of the process also was assessed. The iterative steps in this procedure led to a reduction in the variance between the judges’ratings. All seven judges reported confidence in the final standards. The mean time commitment for each judge was 4.7 work hours. This model for setting performance standards successfully set useful standards for this exam, was practical, and can be utilized to set performance standards for other standardized patient examinations.