Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters

Abstract

ImportanceIn cardiac device implant training, there is no common system to objectively assess trainees’ ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators’ early learning curve.ObjectiveTo assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT).Design, Setting, and ParticipantsIn this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022.InterventionTraining with PBP vs SBT.Main Outcome and MeasuresThe primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined.ResultsBaseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees.Conclusions and RelevanceIn this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications.Trial RegistrationClinicalTrials.gov Identifier: NCT05952908