Frame of Reference Training for Assessment Centers: Effects on Interrater Reliability When Rating Behaviors and Ability Traits

Abstract

Assessment centers have been widely criticized on the basis of measurement problems. The present study sought to present a methodological piece on the extent to which Frame of Reference (FOR) training would increase the interrater reliability associated with assessment center ratings provided by non-psychologist assessors. Five managerial assessors (with no psychological training) rated the behavior and the ability traits of a contrived participant on the basis of behaviors described in two alternative vignettes (detailing critical incidents of job performance). The ratings were obtained both before and following FOR training. It was found that agreement among assessors on their assessment of both behaviors and traits increased subsequent to the FOR training procedure. The implications of increasing the precision associated with assessment center ratings are discussed.