Multi-Content Interaction Network for Few-Shot Segmentation

Abstract

Few-Shot Segmentation (FSS) poses significant challenges due to limited support images and large intra-class appearance discrepancies. Most existing approaches focus on aligning the support-query correlations from the same layer of the frozen backbone while neglecting the bias between different tasks and different layers. In this paper, we propose a Multi-Content Interaction Network (MCINet) to remedy these issues by fully exploiting and interacting with the different contextual information contained in distinct branches. Specifically, MCINet improves FSS from three perspectives: 1) boosting the query representations through incorporating the independent information from another learnable branch into the features from the frozen backbone, 2) enhancing the support-query correlations by exploiting both the same-layer and adjacent-layer features, and 3) refining the predicted results with a multi-scale mask prediction strategy. Experiments on three benchmarks demonstrate that our approach reaches SOTA performances and outperforms the best competitors with many desirable advantages, especially on the challenging COCO dataset. Code will be released at: https://github.com/chenhao-zju/mcinet