Deep Learning in Surgical Workflow Analysis: A review

Abstract

<p>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible</p> <p><br></p> <p><br></p> <p><em>Objective</em>: In the last two decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations at different semantic levels. This information is necessary for developing context-aware intelligent systems, which can assist the physicians during operations, eval- uate procedures afterward or help the management team to effectively utilize the operating room. The objective is to extract reliable patterns from surgical data for the robust estimation of surgical ac- tivities performed during operations. This paper reviews the state- of-the-art methods published after 2018 for the surgical workflow analysis using deep learning methods. <em>Methods</em>: Three databases, IEEE Xplore, Scopus, and PubMed are searched, and additional studies are added through a manual search. After the database search, 343 studies are screened and a total of 41 studies are selected for this review. <em>Conclusion</em>: Utilizing temporal information is essential for recognizing current surgical actions. Contemporary methods used mainly RNNs, hierarchical CNNs models, and Transformers to preserve long-distance temporal relations. The lack of large publicly available datasets for various procedures is a great challenge for the development of new and robust models. As supervised learning strategies used to show proof-of-concept, self-supervised, semi-supervised, or active learning methods are used to mitigate dependency on annotated data. <em>Significance</em>: The present study provides a comprehensive review of recent methods in surgical workflow analysis, summarizes commonly used architectures, datasets, and discusses challenges.</p>