Objective assessment of intraoperative skills for robot-assisted partial nephrectomy (RAPN)

Abstract

Abstract Background RAPN training usually takes place in-vivo and methods vary across countries/ institutions. No common system exists to objectively assess trainee capacity to perform RAPN at predetermined performance levels prior to in-vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. Objective The authors sought to examine the reliability, construct and discriminative validity of objective intraoperative performance metrics which best characterize optimal and suboptimal performance of a reference approach for training novice RAPN surgeons. Design, setting, and participants: Seven Novice and 9 Experienced RAPN surgeons videorecorded one or two independently performed RAPN procedures in the human. The videos were anonymized and two experienced urology surgeons were trained to reliably score RAPN performance, using previously developed metrics. The assessors were blinded to the performing surgeon, hospital and surgeon group. They independently scored surgeon RAPN performance. Novice and Experienced group performance scores were compared for procedure steps completed and errors made. Each group was divided at the median for Total Errors score, and subgroup scores (i.e., Novice HiErrs and LoErrs, Experienced HiErrs and LoErrs) were compared. Results The mean inter-rater reliability (IRR) for scoring was 0.95 (range 0.84–1). Compared with Novices, Experienced RAPN surgeons made 69% fewer procedural Total Errors. This difference was accentuated when LoErr Expert RAPN surgeon’s performance was compared with the HiErrs Novice RAPN surgeon’s performance with an observed 170% fewer Total Errors. GEARS showed poor reliability (Mean IRR = 0.44; range 0.0–0.8), for scoring RAPN surgical performance. Conclusions The RAPN procedure metrics reliably distinguish Novice and Experienced surgeon performances. They further differentiated performance levels within a group with similar experience. Reliable and valid metrics will underpin quality-assured novice RAPN surgical training.