Extra-abdominal trocar and instrument detection for enhanced surgical workflow understanding

Abstract

Abstract Purpose Video-based intra-abdominal instrument tracking for laparoscopic surgeries is a common research area. However, the tracking can only be done with instruments that are actually visible in the laparoscopic image. By using extra-abdominal cameras to detect trocars and classify their occupancy state, additional information about the instrument location, whether an instrument is still in the abdomen or not, can be obtained. This can enhance laparoscopic workflow understanding and enrich already existing intra-abdominal solutions. Methods A data set of four laparoscopic surgeries recorded with two time-synchronized extra-abdominal 2D cameras was generated. The preprocessed and annotated data were used to train a deep learning-based network architecture consisting of a trocar detection, a centroid tracker and a temporal model to provide the occupancy state of all trocars during the surgery. Results The trocar detection model achieves an F1 score of $$95.06\pm 0.88\%$$ 95.06 ± 0.88 % . The prediction of the occupancy state yields an F1 score of $$89.29\pm 5.29\%$$ 89.29 ± 5.29 % , providing a first step towards enhanced surgical workflow understanding. Conclusion The current method shows promising results for the extra-abdominal tracking of trocars and their occupancy state. Future advancements include the enlargement of the data set and incorporation of intra-abdominal imaging to facilitate accurate assignment of instruments to trocars.