Attention-based spatial–temporal neural network for accurate phase recognition in minimally invasive surgery: feasibility and efficiency verification

Abstract

Abstract Laparoscopic surgery, as a representative minimally invasive surgery (MIS), is an active research area of clinical practice. Automatic surgical phase recognition of laparoscopic videos is a vital task with the potential to improve surgeons’ efficiency and has gradually become an integral part of computer-assisted intervention systems in MIS. However, the performance of most methods currently employed for surgical phase recognition is deteriorated by optimization difficulties and inefficient computation, which hinders their large-scale practical implementation. This study proposes an efficient and novel surgical phase recognition method using an attention-based spatial–temporal neural network consisting of a spatial model and a temporal model for accurate recognition by end-to-end training. The former subtly incorporates the attention mechanism to enhance the model’s ability to focus on the key regions in video frames and efficiently capture more informative visual features. In the temporal model, we employ independently recurrent long short-term memory (IndyLSTM) and non-local block to extract long-term temporal information of video frames. We evaluated the performance of our method on the publicly available Cholec80 dataset. Our attention-based spatial–temporal neural network purely produces the phase predictions without any post-processing strategies, achieving excellent recognition performance and outperforming other state-of-the-art phase recognition methods.