STEREOSCOPIC AUGMENTED REALITY FOR INTRAOPERATIVE GUIDANCE IN ROBOTIC SURGERY

Abstract

Augmented reality (AR) technology is increasingly adopted in the surgical field and recently it has been also introduced in robotic-assisted urologic surgery. This work describes the design and development of an AR intraoperative guide system with stereoscopic visualization (SAR, stereoscopic augmented reality) for the Da Vinci surgical robot. As a major novelty, the developed SAR system allows the surgeon to have the virtual 3D model of patient anatomy superimposed on the real field, without losing the stereoscopic view of the operative field. The workflow starts with the 3D model generation of the anatomical district of interest for surgery, from patient diagnostic imaging. Then, the 3D model is uploaded in the developed SAR application, navigated using a 3D space mouse, and superimposed to the operative field using computer vision algorithms. The SAR system was tested during 30 robot-assisted surgeries, including 20 partial nephrectomies, 1 kidney explant, and 9 radical prostatectomies. The SAR guidance system received overall great appreciation from surgeons and helped in localizing hidden structures, such as arteries or tumoral masses, increasing the understanding of surgical anatomy with depth perception, and facilitating intraoperative navigation. Future efforts will be addressed to improve the automatic superimposition of digital 3D models on the intraoperative view.