Landsat-8 to Sentinel-2 Satellite Imagery Super-Resolution-Based Multiscale Dilated Transformer Generative Adversarial Networks

Abstract

Image super-resolution (SR) techniques can improve the spatial resolution of remote sensing images to provide more feature details and information, which is important for a wide range of remote sensing applications, including land use/cover classification (LUCC). Convolutional neural networks (CNNs) have achieved impressive results in the field of image SR, but the inherent localization of convolution limits the performance of CNN-based SR models. Therefore, we propose a new method, namely, the dilated Transformer generative adversarial network (DTGAN) for the SR of multispectral remote sensing images. DTGAN combines the local focus of CNNs with the global perspective of Transformers to better capture both local and global features in remote sensing images. We introduce dilated convolutions into the self-attention computation of Transformers to control the network’s focus on different scales of image features. This enhancement improves the network’s ability to reconstruct details at various scales in the images. SR imagery provides richer surface information and reduces ambiguity for the LUCC task, thereby enhancing the accuracy of LUCC. Our work comprises two main stages: remote sensing image SR and LUCC. In the SR stage, we conducted comprehensive experiments on Landsat-8 (L8) and Sentinel-2 (S2) remote sensing datasets. The results indicate that DTGAN generates super-resolution (SR) images with minimal computation. Additionally, it outperforms other methods in terms of the spectral angle mapper (SAM) and learned perceptual image patch similarity (LPIPS) metrics, as well as visual quality. In the LUCC stage, DTGAN was used to generate SR images of areas outside the training samples, and then the SR imagery was used in the LUCC task. The results indicated a significant improvement in the accuracy of LUCC based on SR imagery compared to low-resolution (LR) LUCC maps. Specifically, there were enhancements of 0.130 in precision, 0.178 in recall, and 0.157 in the F1-score.