Assessment tool validation and technical skill improvement in the simulation of the Norwood operation using three-dimensional-printed heart models

Abstract

Abstract OBJECTIVES The hands-on surgical training course utilizes 3-dimensional (3D)-printed heart models to simulate complex congenital heart operations. This study aimed to validate a model and assessment tool in the simulation of 2 techniques of the Norwood operation and investigate whether technical performance improves following rehearsal with or without proctor presence. METHODS Five ‘experienced’ and 5 ‘junior’ surgeons performed 2 techniques of the Norwood operation on 3D-printed models of hypoplastic left heart syndrome. Performances were retrospectively assessed by 10 raters with varying experience in congenital heart surgery (CHS) (Medical Doctorate versus non-Medical Doctorate). Assessments were made with the procedure-specific Hands-On Surgical Training-CHS tool. Results were analysed for technical performance and rater consistency. Following validation, 30 surgeons (24 with proctor guidance and 6 with training videos and objective feedback only) simulated the Norwood operation twice. RESULTS Performance scores were consistently higher for experienced surgeons and raters discriminated clearly between the experienced and junior surgeons (P ≤ 0.001). The hands-on surgical training-CHS tool showed high inter-rater (0.86) and intra-rater (0.80) reliability among all raters. Scores for both experienced and junior surgeons were highly consistent across all raters, with no statistically significant difference (P = 0.50). All surgeons successfully performed the Norwood operation. Sixty attempts were scored in total. Eighty-seven percentage (26/30) of surgeons’ scores (mean: attempt 1 = 92, attempt 2 = 104) and times [mean: attempt 1 = 1:22:00, attempt 2 = 1:08:00 (h:mm:ss)] improved between the 2 attempts by 9% and 15% respectively (P ≤ 0.001). Total scores of all surgeons in the non-proctored subgroup (6) improved by 15% on average (mean: attempt 1 = 86, attempt 2 = 105, P = 0.002). CONCLUSIONS Procedure-specific assessment tools can be developed to evaluate technical performance for complex CHS simulation and be performed reliably by non-expert raters. Rehearsal both under supervision and independently leads to technical skill improvement further supporting its value in CHS training.