Strategies in Comparison of Methods for Scoring Patient Managementproblems

Abstract

The study was designed to determine whether alternative scoring strategies result in improved measurement properties of patient management problems (PMP). For each of 16 PMPs used in a certifying examination, thefollowingscoringsystems were developed: proficiency, efficiency, select, omit, data gathering, therapy, absolute (dichotomouspass/fail), goal-oriented, and an empiric expert score. Scores with each system were developedfor 4,590firsttime takers and compared to two external criterion measures, namely, the examinees' "clinical-competence" rating by their program director, and the examinees'membership in a "high" or "low" criterion status group. The results demonstrated high correlations between all scoring systems and high correlations with simultaneous multiple-choice (MCQ) scores. Each showed moderate correlation with the validity measures. Using regression analysis, none of the PMPs coresprovided significant unique incremental validity not provided by MCQ scores. The implications of these data in the use of PMPs in certifying or licensure examinations are discussed.