LPAM: A lightweight medical segmentation network based on Mamba improved by prompt attention

Abstract

AbstractPresently, State Space Models (SSMs), including frameworks like Mamba, have been incorporated into the realm of computer vision. These models not only sustain remote interactions and encapsulate global semantic information effectively, but also preserve linear computational complexity, offering a balance between performance and computational efficiency. Given that Mamba inherently adheres to the principle of selectivity when constructing sequence models, the goal is to further unleash the potential of Mamba through this innovative combination of convolution and self‐attention, improve accuracy and minimize the number of parameters while achieving linear complexity. Mamba is employed as an encoder to distill semantic information from the image, and it is supplemented with convolutional blocks, thereby conserving the details of the image. Concurrently, embedding prompts at a deeper level enhances its adaptability to cater to diverse requirements. Lastly, a bidirectional attention mechanism is incorporated for inference, striving to retain both global connections and local details to the maximum extent. This culminates in a novel, lightweight medical segmentation model. Exhaustive experiments were executed on six public datasets. The empirical results show that the proposed model exhibits competitive performance in medical image segmentation tasks.