Validation of an assessment tool for pre-operative EVAR planning

Abstract

Introduction: Current methods of teaching endovascular aortic aneurysm repair (EVAR) planning involve non-criteria-based observations that lack validity. The primary aim of this study was to validate an EVAR Planning Objective Structured Assessment of Skill (EpOSAS) tool for the assessment of pre-operative EVAR planning skills. Methods: Content analysis was performed in order to inform the formulation of EpOSAS domains. Thirty-five participants planned two cases of infra-renal abdominal aortic aneurysm for EVAR, using the OsiriX 7 platform. EVAR planning measurements, with accompanying screenshots, were uploaded onto an electronic data collection sheet. Performance was assessed by three blinded assessors using the EpOSAS tool. Construct and concurrent validity were evaluated. Results: Inter-observer reliability for the three assessors for total EpOSAS scores was high (Cronbach’s α 0.89). There were statistically significant differences in total EpOSAS scores between the different experience groups, demonstrating construct validity (Novice (5.3, IQR 5-5.3), Intermediate (15.3, IQR 14.8-16.8) and Experts (17.5, IQR 17-17.7), p<0.001). A statistically significant correlation was found between total EpOSAS scores and percentage error in measurements, demonstrating concurrent validity (Spearman’s rank correlation coefficient: R=-0.250, p<0.001). Receiver-operator characteristics analysis established a cut-off point of 16 out of 18 for determining competence. Conclusion: We have developed and validated a tool that can be used for the assessment of pre-operative EVAR planning skills.