Promoting Longitudinal and Developmental Computer-Based Assessments of Clinical Reasoning: Validity Evidence for a Clinical Reasoning Mapping Exercise

Abstract

Abstract Purpose Clinical reasoning is vitally important for practitioners across the health professions. However, the assessment of clinical reasoning remains a significant challenge. Combined with other assessment methods, non–workplace-based assessment can increase opportunities to address multiple components of clinical reasoning, evaluate growth, and foster learning, but tools with validity evidence to assess clinical reasoning outside the workplace are scare. This study examined validity evidence for a novel clinical reasoning mapping exercise (CResME). Method Data include CResME performance scores from 120 third-year medical students at the University of Central Florida for 3 topics in May 2022. Each CResME was scored by 2 physician raters based on a scoring rubric that included a combined diagnosis and sequence score. Descriptive statistics were used to examine trends in scores. The authors gathered validity evidence for response process, internal structure, and relations to other variables. Results The overall mean (SD) score across cases was 66 (29). Internal consistency reliability of cases (Cronbach α) ranged from 0.75 to 0.91. The Phi and G coefficients were 0.45 and 0.56, respectively. Students accounted for 10% of the total variance, indicating the ability to differentiate high and low clinical reasoning skills; the interaction between learner and case accounted for 8.1% of the variance, demonstrating case specificity. There was a moderate correlation between the overall CResME scores and the mean overall score of patient encounter notes from an objective structured clinical examination performed at the end of the third year (0.46; P = .001). Significant associations were also found between the CResME scores and subject exam scores. Conclusions The CResME can be used to facilitate the assessment of clinical reasoning, supporting the developmental progress of learners throughout the curriculum. Future research is needed to gather validity evidence for CResMEs with different learners across different settings and formats.