Deep Learning–Based Real-Time Ureter Identification in Laparoscopic Colorectal Surgery

Abstract

BACKGROUND: Iatrogenic ureteral injury is a serious complication of abdominopelvic surgery. Identifying the ureters intraoperatively is essential to avoid iatrogenic ureteral injury. We developed a model that may minimize this complication. IMPACT OF INNOVATION: We applied a deep learning–based semantic segmentation algorithm to the ureter recognition task and developed a deep learning model called UreterNet. This study aimed to verify whether the ureters could be identified in videos of laparoscopic colorectal surgery. TECHNOLOGY, MATERIALS, AND METHODS: Semantic segmentation of the ureter area was performed using a convolutional neural network–based approach. Feature Pyramid Networks were used as the convolutional neural network architecture for semantic segmentation. Precision, recall, and the Dice coefficient were used as the evaluation metrics in this study. PRELIMINARY RESULTS: We created 14,069 annotated images from 304 videos, with 9537, 2266, and 2266 images in the training, validation, and test data sets, respectively. Concerning ureter recognition performance, the precision, recall, and Dice coefficient for the test data were 0.712, 0.722, and 0.716, respectively. Regarding the real-time performance on recorded videos, it took 71 milliseconds for UreterNet to infer all pixels corresponding to the ureter from a single still image and 143 milliseconds to output and display the inferred results as a segmentation mask on the laparoscopic monitor. CONCLUSIONS: UreterNet is a noninvasive method for identifying the ureter in videos of laparoscopic colorectal surgery and can potentially improve surgical safety. FUTURE DIRECTIONS: Although this deep learning model could lead to the development of an image-navigated surgical system, it is necessary to verify whether UreterNet reduces the occurrence of iatrogenic ureteral injury.