Affiliation:
1. School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China
Abstract
Convolutional neural networks (CNNs) have received significant attention for change detection (CD) on multimodal remote sensing images, but they struggle to capture global cues due to the locality of convolution operations. In contrast, the transformer can learn global semantic information by dividing the input image into patches, adding position encodings, and utilizing the self-attention mechanism. Motivated by this, we propose mSwinUNet, a novel end-to-end multi-modal model with swin-transformer-based and U-shaped siamese network architectures for supervised CD using Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 Multispectral Imager (MSI) data. mSwinUNet contains multi-modal encoder with difference module, bottleneck, and fused decoder, and all of them are based on swin transformer. Firstly, tokenized multi-modal bitemporal image patches are fed into multiple Siamese encoder branches to extract multi-level multi-modal difference feature maps in parallel. Subsequently, the last level multi-modal difference maps are fused to generate the smallest scale change map in the bottleneck. Then, the hierarchical decoder incorporates patch expansion and fusion operations to fuse multi-scale difference and change maps, effectively recuperating the details of the change information. Finally, the last patch expansion and a linear projection are applied to output the final change map, which preserves the identical spatial resolution as the input image. Extensive experiments have shown that mSwinUNet outperforms several the state-of-the-art multi-modal CD methods on OSCD dataset and the corresponding Sentinel-1 SAR data.
Subject
Artificial Intelligence,General Engineering,Statistics and Probability
Reference20 articles.
1. Earthquake damage assessmentof buildings using vhr optical and sar imagery;Brunner;IEEETransactions on Geoscience and Remote Sensing,2010
2. Remote sensing image change detectionwith transformers;Chen;IEEE Transactions on Geoscience and RemoteSensing,2021
3. Dasnet: Dual attentive fully convolutional siamese networksfor change detection in high-resolution satellite images;Chen;IEEEJournal of Selected Topics in Applied Earth Observations and RemoteSensing,2020
4. The influence of polarimetricparameters and an object-based approach on land cover classificationin coastal wetlands;Chen;Remote Sensing,2014
5. Fusing multi-modal data forsupervised change detection;Ebel;The International Archives of thePhotogrammetry, Remote Sensing and Spatial Information Sciences,2021