Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs

Abstract

BACKGROUND: The American Board of Surgery has endorsed competency-based education as vital to the assessment of surgical training. From 2018 to 2020, a national pilot study was conducted at 28 general surgery programs to evaluate feasibility of implementing entrustable professional activities (EPAs) for 5 common general surgical conditions. ACGME core competency Milestones were also rated for each resident by program clinical competency committees. This study aimed to evaluate the validity of general surgery EPAs compared with Milestones. STUDY DESIGN: Prospectively collected, de-identified EPA Pilot Study data were analyzed. EPAs studied were EPA-1 (gallbladder), EPA-2 (inguinal hernia), EPA-3 (right lower quadrant pain), EPA-4 (trauma), and EPA-5 (consult). Variables abstracted included levels of EPA entrustment (1 to 5) and corresponding ACGME Milestone subcompetency ratings (1 to 5) for the same study cycle. Spearman’s correlations were used to evaluate the relationship between summative EPA scores and corresponding Milestone ratings. RESULTS: A total of 493 unique residents received a summative entrustment decision. EPA summative entrustment scores had moderate-to-strong positive correlation with mapped Milestone subcompetencies, with median rho value of 0.703. Among operation-focused EPAs, median rho values were similar between EPA-1 (0.688) and EPA-2 (0.661), but higher for EPA-3 (0.833). EPA-4 showed a strong positive correlation with diagnosis and communication competencies (0.724), whereas EPA-5, mapped to the most Milestone subcompetencies, had the lowest median rho value (0.455). CONCLUSIONS: Moderate-to-strong positive correlation was noted between EPAs and patient care, medical knowledge, and communication Milestones. These findings support the validity of EPAs in general surgery and suggest that EPA assessments can be used to inform Milestone ratings by clinical competency committees.