Surgeon Assistive Augmented Reality Model with the use of Endoscopic Camera for Line of Vision Calculation

Abstract

In this paper, the authors describe a surgeon assistive Augmented Reality (AR) model for endoscopic procedures. They analyze the main parts of the model and the processes that need to be established such as, the registration of the patient, the segmentation of medical data, their 3D reconstruction, and the detection of endoscopic instruments and the camera. The authors present two graphical user interfaces, build to serve the needs of segmentation, navigation, and visualization of the final intra-operative scene. By using preoperative data of the patient (MRI-CT) and image processing techniques, the authors can provide a unique view of the surgical scene. The potentials and the advantages of endoscopic-robotic surgeries nowadays can be improved. Augmented surgery scenes with information about the patients underline structures, enables wider situation awareness, precision, and confidence.