Oral Examination versus Simulation-Based Assessment in Assessing Patient Care Competencies for Emergency Medicine Residents

Abstract

<b><i>Background:</i></b> Assessment is essential for any accreditation process in the medical field. If a candidate passes a high-stakes assessment, they can work independently. While oral examinations are common, given the complexity of clinical competencies, such an approach may not be the most effective assessment method. A form of performance-based assessment, such as a simulation, may be beneficial in this context. <b><i>Objectives:</i></b> This study aims to determine whether the results of oral examinations match those of simulation-based assessments when both modalities are used to evaluate residents’ performance in scenarios featuring similar content. It also seeks to determine whether oral examinations under- or overestimate residents’ competencies concerning patient care when compared to their simulation performance. <b><i>Methods:</i></b> This is a cross-sectional, single-centre study. Emergency medicine residents underwent an oral examination and completed a simulation-based assessment. Standardized scenarios were used to assess the residents’ emergency medicine competencies. A global rating scale was used to rate participants’ performance in each assessment modality. <b><i>Results:</i></b> There was a moderate positive correlation between oral examination and simulation-based assessment results (<i>r</i> = 0.699, <i>p</i> &#x3c; 0.05, <i>n</i> = 28). A paired <i>t</i> test indicates that the oral examination overestimates residents’ competency compared to the simulation-based assessment; the mean difference is 0.26 (confidence interval: 0.041–0.493). <b><i>Conclusions:</i></b> Emergency medicine residents whose knowledge was assessed at the “know-how” level of Miller’s pyramid in the oral examination were not necessarily able to move up to the level of “show-how” by demonstrating the ability to apply their knowledge in the simulation-based assessment. The findings of this study confirm that simulation-based assessments should be an essential aspect of high-stakes examinations intended to determine residents’ different clinical competencies.