The Validity of Credentialing Examinations

Abstract

Job relevance, along with adverse impact, are major validity concerns relative to credentialing examinations. Although it is generally recognized that job relevance is facilitated by utilizing a job analysis in developing the examination blueprint, test developers typically are not clear about the legal expectations of this approach and the specific procedures that must be followed to ensure job relatedness in developing and scoring examinations. This article recommends that examination blueprints utilize a competency model based on three dimensions: the job functions assigned, the level of expertise expected given the content and resources typically available, and the specific context or situation in which the job must be performed. It also presents the critical scenario technique for generating items for examinations and discusses weighted scoring and error-pattern scaling techniques for reporting results. It is argued that a credentialing examination based on a competency model, which in turn is derived from job analysis data, would ensure defensible job relevance to such examinations.