Affiliation:
1. Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
2. Shandong Non-Metallic Materials Institute, Linyi 250031, China
3. School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing 401331, China
Abstract
The main challenge of scene classification is to understand the semantic context information of high-resolution remote sensing images. Although vision transformer (ViT)-based methods have been explored to boost the long-range dependencies of high-resolution remote sensing images, the connectivity between neighboring windows is still limited. Meanwhile, ViT-based methods commonly contain a large number of parameters, resulting in a huge computational consumption. In this paper, a novel lightweight dual-branch swin transformer (LDBST) method for remote sensing scene classification is proposed, and the discriminative ability of scene features is increased through combining a ViT branch and convolutional neural network (CNN) branch. First, based on the hierarchical swin transformer model, LDBST divides the input features of each stage into two parts, which are then separately fed into the two branches. For the ViT branch, a dual multilayer perceptron structure with a depthwise convolutional layer, termed Conv-MLP, is integrated into the branch to boost the connections with neighboring windows. Then, a simple-structured CNN branch with maximum pooling preserves the strong features of the scene feature map. Specifically, the CNN branch lightens the LDBST, by avoiding complex multi-head attention and multilayer perceptron computations. To obtain better feature representation, LDBST was pretrained on the large-scale remote scene classification images of the MLRSN and RSD46-WHU datasets. These two pretrained weights were fine-tuned on target scene classification datasets. The experimental results showed that the proposed LDBST method was more effective than some other advanced remote sensing scene classification methods.
Funder
Natural Science Foundation of Chongqing
National Natural Science Foundation of China
Science and Technology Research Program of Chongqing Municipal Education Commission
Cooperation project between Chongqing Municipal undergraduate universities and institutes affiliated to the Chinese Academy of Sciences
Subject
General Earth and Planetary Sciences
Reference44 articles.
1. Saliency-guided unsupervised feature learning for scene classification;Zhang;IEEE Trans. Geosci. Remote Sens.,2014
2. Fan, Z., Yu, J.-G., Liang, Z., Ou, J., Gao, C., Xia, G.-S., and Li, Y. (2020, January 14–19). FGN: Fully guided network for few-shot instance segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.
3. Remote sensing image retrieval using convolutional neural network features and weighted distance;Ye;IEEE Geosci. Remote Sens. Lett.,2018
4. Dual-Aligned Oriented Detector;Cheng;IEEE Trans. Geosci. Remote Sens.,2022
5. Wu, C., Du, B., and Zhang, L. (2022). Fully convolutional change detection framework with generative adversarial network for unsupervised, weakly supervised and regional supervised change detection. arXiv.
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献