GFRNet: Rethinking the global contexts extraction in medical images segmentation through matrix factorization and self‐attention

Abstract

AbstractDue to the large fluctuations of the boundaries and internal variations of the lesion regions in medical image segmentation, current methods may have difficulty capturing sufficient global contexts effectively to deal with these inherent challenges, which may lead to a problem of segmented discrete masks undermining the performance of segmentation. Although self‐attention can be implemented to capture long‐distance dependencies between pixels, it has the disadvantage of computational complexity and the global contexts extracted by self‐attention are still insufficient. To this end, the authors propose the GFRNet, which resorts to the idea of low‐rank matrix factorization by forming global contexts locally to obtain global contexts that are totally different from contexts extracted by self‐attention. The authors effectively integrate the different global contexts extract by self‐attention and low‐rank matrix factorization to extract versatile global contexts. Also, to recover the spatial contexts lost during the matrix factorization process and enhance boundary contexts, the authors propose the Modified Matrix Decomposition module which employ depth‐wise separable convolution and spatial augmentation in the low‐rank matrix factorization process. Comprehensive experiments are performed on four benchmark datasets showing that GFRNet performs better than the relevant CNN and transformer‐based recipes.