Anatomy Segmentation in Laparoscopic Surgery: Comparison of Machine Learning and Human Expertise

Abstract

StructuredAbstractBackgroundLack of anatomy recognition represents a clinically relevant risk in abdominal surgery. Machine learning (ML) methods can help identify visible patterns and risk structures, however, their practical value remains largely unclear.Materials and MethodsBased on a novel dataset of 13195 laparoscopic images with pixel-wise segmentations of eleven anatomical structures, we developed specialized segmentation models for each structure and combined models for all anatomical structures using two state-of-the-art model architectures (DeepLabv3 and SegFormer), and compared segmentation performance of algorithms to a cohort of 28 physicians, medical students, and medical laypersons using the example of pancreas segmentation.ResultsMean Intersection-over-Union for semantic segmentation of intraabdominal structures ranged from 0.28 to 0.83 and from 0.23 to 0.77 for the DeepLabv3-based structure-specific and combined models, and from 0.31 to 0.85 and from 0.26 to 0.67 for the SegFormer-based structure-specific and combined models, respectively. Both the structure-specific and the combined DeepLabv3-based models are capable of near-real-time operation, while the SegFormer-based models are not. All four models outperformed at least 26 out of 28 human participants in pancreas segmentation.ConclusionsThese results demonstrate that ML methods have the potential to provide relevant assistance in anatomy recognition in minimally-invasive surgery in near-real-time. Future research should investigate the educational value and subsequent clinical impact of respective assistance systems.HighlightsMachine learning models to reduce surgical risks that precisely identify 11 anatomical structures: abdominal wall, colon, intestinal vessels (inferior mesenteric artery and inferior mesenteric vein with their subsidiary vessels), liver, pancreas, small intestine, spleen, stomach, ureter and vesicular glandsLarge training dataset of 13195 real-world laparoscopic images with high-quality anatomy annotationsSimilar performance of individual segmentation models for each structure and combined segmentation models in identifying intraabdominal structures, and similar segmentation performance of DeepLabv3-based and SegFormer-based modelsDeepLabv3-based models are capable of near-real-time operation while SegFormer-based models are not, but SegFormer-based models outperform DeepLabv3-based models in terms of accuracy and generalizabilityAll models outperformed at least 26 out of 28 human participants in pancreas segmentation, demonstrating their potential for real-time assistance in recognizing anatomical landmarks during minimally-invasive surgery.