Assessing Diagnostic Reasoning Using a Standardized Case-Based Discussion

Abstract

Background: Development of diagnostic reasoning (DR) is fundamental to medical students’ training, but assessing DR is challenging. Several written assessments focus on DR but lack the ability to dynamically assess DR. Oral assessment formats have strengths but have largely lost favour due to concerns about low reliability and lack of standardization. Medical schools and specialist medical colleges value many forms of oral assessment (eg, long case, Objective Structured Clinical Examination [OSCE], viva voce) but are increasingly searching for ways in which to standardize these formats. We sought to develop and trial a Standardized Case-Based Discussion (SCBD), a highly standardized and interactive oral assessment of DR. Methods: Two initial cohorts of medical students (n = 319 and n = 342) participated in the SCBD as part of their assessments. All students watch a video trigger (based on an authentic clinical case) and discuss their DR with an examiner for 15 minutes. Examiners probe students’ DR and assess how students respond to new standardized clinical information. An online examiner training module clearly articulates expected student performance standards. We used student achievement and student and examiner perceptions to gauge the performance of this new assessment form over 2 implementation years. Results: The SCBD was feasible to implement for a large student cohort and was acceptable to students and examiners. Most students and all examiners agreed that the SCBD discussion provided useful information on students’ DR. The assessment had acceptable internal consistency, and the associations with other assessment formats were small and positive, suggesting that the SCBD measures a related, yet novel construct. Conclusions: Rigorous, standardized oral assessments have a place in a programme of assessment in initial medical training because they provide opportunities to explore DR that are limited in other formats. We plan to incorporate an SCBD into our clinical assessments for the first year of clinical training, where teaching and assessing basic DR is emphasized. We will also explore further examiners’ understanding of and approach to assessing DR.