Refined Division Features Based on Transformer for Semantic Image Segmentation

Abstract

Transformer can build global relationships between pixels and enhance pixel representation. The existing methods only establish the context relationship from the whole image but will reduce the representation between the category areas. In addition, the existing methods based on the transformer self-attention do not combine the advantages of convolution and transformer, resulting in more calculation parameters of the model. In order to solve these two problems, this paper proposes to enhance the segmentation accuracy and performance by enhancing the relationship between image-level regions and the relationship between semantic level pixels. First, we design a refined division feature (RDF) module to enhance the channel representation and thus the same locale representation. Second, we design a transformer based on convolution (CTrans), which first computes the relationship between similar pixels and enhances the pixel representation. Then, the feature map is compressed and enriched to reduce the computational load of CTrans, and finally the relationship between pixels is established from a global perspective. We design a refined division feature module based on transformer for semantic image segmentation (RFT) model combining RDF and CTrans module. The experimental results show that the mIoU result of our method in Cityscapes test data set is 81.9%, and the model parameter is 64.6M, which is superior to other methods in terms of data. In addition, we conducted visualization experiments with Cityscapes and Pascal voc 2012 datasets with other methods, and the results showed that our method was superior to other methods.