Development of an inside-out augmented reality technique for neurosurgical navigation

Abstract

OBJECTIVE With the advancement of 3D modeling techniques and visualization devices, augmented reality (AR)–based navigation (AR navigation) is being developed actively. The authors developed a pilot model of their newly developed inside-out tracking AR navigation system. METHODS The inside-out AR navigation technique was developed based on the visual inertial odometry (VIO) algorithm. The Quick Response (QR) marker was created and used for the image feature–detection algorithm. Inside-out AR navigation works through the steps of visualization device recognition, marker recognition, AR implementation, and registration within the running environment. A virtual 3D patient model for AR rendering and a 3D-printed patient model for validating registration accuracy were created. Inside-out tracking was used for the registration. The registration accuracy was validated by using intuitive, visualization, and quantitative methods for identifying coordinates by matching errors. Fine-tuning and opacity-adjustment functions were developed. RESULTS ARKit-based inside-out AR navigation was developed. The fiducial marker of the AR model and those of the 3D-printed patient model were correctly overlapped at all locations without errors. The tumor and anatomical structures of AR navigation and the tumors and structures placed in the intracranial space of the 3D-printed patient model precisely overlapped. The registration accuracy was quantified using coordinates, and the average moving errors of the x-axis and y-axis were 0.52 ± 0.35 and 0.05 ± 0.16 mm, respectively. The gradients from the x-axis and y-axis were 0.35° and 1.02°, respectively. Application of the fine-tuning and opacity-adjustment functions was proven by the videos. CONCLUSIONS The authors developed a novel inside-out tracking–based AR navigation system and validated its registration accuracy. This technical system could be applied in the novel navigation system for patient-specific neurosurgery.