Pan-Sharpening Network of Multi-Spectral Remote Sensing Images Using Two-Stream Attention Feature Extractor and Multi-Detail Injection (TAMINet)
-
Published:2023-12-24
Issue:1
Volume:16
Page:75
-
ISSN:2072-4292
-
Container-title:Remote Sensing
-
language:en
-
Short-container-title:Remote Sensing
Author:
Wang Jing12, Miao Jiaqing1ORCID, Li Gaoping1, Tan Ying3ORCID, Yu Shicheng4, Liu Xiaoguang1, Zeng Li1, Li Guibing35
Affiliation:
1. School of Mathematics, Southwest Minzu University, Chengdu 610041, China 2. School of Information and Business Management, Chengdu Neusoft University, Chengdu 611844, China 3. Key Laboratory of Computer System, State Ethnic Affairs Commission, College of Computer Science and Engineering, Southwest Minzu University, Chengdu 610041, China 4. School of Big Data and Artificial Intelligence, Chengdu Technological University, Chengdu 611730, China 5. School of Electrical Engineering, Southwest Jiaotong University, Chengdu 611756, China
Abstract
Achieving a balance between spectral resolution and spatial resolution in multi-spectral remote sensing images is challenging due to physical constraints. Consequently, pan-sharpening technology was developed to address this challenge. While significant progress was recently achieved in deep-learning-based pan-sharpening techniques, most existing deep learning approaches face two primary limitations: (1) convolutional neural networks (CNNs) struggle with long-range dependency issues, and (2) significant detail loss during deep network training. Moreover, despite these methods’ pan-sharpening capabilities, their generalization to full-sized raw images remains problematic due to scaling disparities, rendering them less practical. To tackle these issues, we introduce in this study a multi-spectral remote sensing image fusion network, termed TAMINet, which leverages a two-stream coordinate attention mechanism and multi-detail injection. Initially, a two-stream feature extractor augmented with the coordinate attention (CA) block is employed to derive modal-specific features from low-resolution multi-spectral (LRMS) images and panchromatic (PAN) images. This is followed by feature-domain fusion and pan-sharpening image reconstruction. Crucially, a multi-detail injection approach is incorporated during fusion and reconstruction, ensuring the reintroduction of details lost earlier in the process, which minimizes high-frequency detail loss. Finally, a novel hybrid loss function is proposed that incorporates spatial loss, spectral loss, and an additional loss component to enhance performance. The proposed methodology’s effectiveness was validated through experiments on WorldView-2 satellite images, IKONOS, and QuickBird, benchmarked against current state-of-the-art techniques. Experimental findings reveal that TAMINet significantly elevates the pan-sharpening performance for large-scale images, underscoring its potential to enhance multi-spectral remote sensing image quality.
Funder
NSFC Sichuan Science and Technology Project Sichuan Provincial Program of Traditional Chinese Medicine Sichuan Science and Technology Program Fundamental Research Funds for the Central Universities, Southwest Minzu University
Subject
General Earth and Planetary Sciences
Reference51 articles.
1. Application of Hyperspectral Remote Sensing for Environment Monitoring in Mining Areas;Zhang;Environ. Earth Sci.,2012 2. Image Fusion Meets Deep Learning: A Survey and Perspective;Zhang;Inf. Fusion,2021 3. Jones, E.G., Wong, S., Milton, A., Sclauzero, J., Whittenbury, H., and McDonnell, M.D. (2020). The Impact of Pan-Sharpening and Spectral Resolution on Vineyard Segmentation through Machine Learning. Remote Sens., 12. 4. Gao, J., Li, J., Su, X., Jiang, M., and Yuan, Q. (2022, January 19–20). Deep Image Interpolation: A Unified Unsupervised Framework for Pansharpening. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), New Orleans, LA, USA. 5. Extracting Spectral Contrast in Landsat Thematic Mapper Image Data Using Selective Principal Component Analysis;Chavez;Photogramm. Eng. Remote Sens.,1989
|
|