A Validity Study of Part III of the National Board Examination

Abstract

The purpose of this study was to investigate the validity of Part III of the National Board Examination, a certifying examination of medical knowledge and patient management abilities. The subjects were 1,866 first-year resident physicians who graduated from Jefferson Medical College between 1970 and 1984. Statistically significant correlations were found between scores on this examination on the one hand and measures of basic and clinical sciences in medical school and Parts I and II of the National Board Examination on the other hand. Also, graduates who were rated high on supervisors' ratings of clinical competence areas in residency obtained higher scores on the examination than those rated low on this scale even when their baseline knowledge (scores on Part II) was controlled by employing analysis of covariance. In addition, assumptions were made that graduates who were offered further residency training and those who pursued broader, less-specialized careers would score higher on this examination. Both assumptions were confirmed. This 15-year study not only provides unique information about the validity of one certifying examination, but it also presents a model that might be used to evaluate other certification tests. Improvements in validity produced by new testing methods, such as computerized administration, could be put into perspective by using similar validity studies.