SeaMAE: Masked Pre-Training with Meteorological Satellite Imagery for Sea Fog Detection
-
Published:2023-08-21
Issue:16
Volume:15
Page:4102
-
ISSN:2072-4292
-
Container-title:Remote Sensing
-
language:en
-
Short-container-title:Remote Sensing
Author:
Yan Haotian1, Su Sundingkai1ORCID, Wu Ming1, Xu Mengqiu1ORCID, Zuo Yihao1, Zhang Chuang1, Huang Bin2
Affiliation:
1. Artificial Intelligence School, Beijing University of Posts and Telecommunications, Beijing 100876, China 2. National Meteorological Center, China Meteorological Administration, Beijing 100081, China
Abstract
Sea fog detection (SFD) presents a significant challenge in the field of intelligent Earth observation, particularly in analyzing meteorological satellite imagery. Akin to various vision tasks, ImageNet pre-training is commonly used for pre-training SFD. However, in the context of multi-spectral meteorological satellite imagery, the initial step of deep learning has received limited attention. Recently, pre-training with Very High-Resolution (VHR) satellite imagery has gained increased popularity in remote-sensing vision tasks, showing the potential to replace ImageNet pre-training. However, it is worth noting that the meteorological satellite imagery applied in SFD, despite being an application of computer vision in remote sensing, differs greatly from VHR satellite imagery. To address the limitation of pre-training for SFD, this paper introduces a novel deep-learning paradigm to the meteorological domain driven by Masked Image Modeling (MIM). Our research reveals two key insights: (1) Pre-training with meteorological satellite imagery yields superior SFD performance compared to pre-training with nature imagery and VHR satellite imagery. (2) Incorporating the architectural characteristics of SFD models into a vanilla masked autoencoder (MAE) can augment the effectiveness of meteorological pre-training. To facilitate this research, we curate a pre-training dataset comprising 514,655 temporal multi-spectral meteorological satellite images, covering the Bohai Sea and Yellow Sea regions, which have the most sea fog occurrence. The longitude ranges from 115.00E to 128.75E, and the latitude ranges from 27.60N to 41.35N. Moreover, we introduce SeaMAE, a novel MAE that utilizes a Vision Transformer as the encoder and a convolutional hierarchical decoder, to learn meteorological representations. SeaMAE is pre-trained on this dataset and fine-tuned for SFD, resulting in state-of-the-art performance. For instance, using the ViT-Base as the backbone, SeaMAE pre-training which achieves 64.18% surpasses from-scratch learning, natural imagery pre-training, and VRH satellite imagery pre-training by 5.53%, 2.49%, and 2.21%, respectively, in terms of Intersection over Union of SFD.
Funder
National Key R&D Program of China
Subject
General Earth and Planetary Sciences
Reference56 articles.
1. The Longmen cloud physics field experiment base, China Meteorological Administration;Liu;J. Trop. Meteorol.,2023 2. Managing weather & fishing safety: Marine meteorology and fishing decision-making from a governance and safety perspective;Finnis;Mar. Policy,2022 3. Guo, X., Wan, J., Liu, S., Xu, M., Sheng, H., and Yasir, M. (2021). A scse-linknet deep learning model for daytime sea fog detection. Remote Sens., 13. 4. Zhu, C., Wang, J., Liu, S., Sheng, H., and Xiao, Y. (2019, January 24–26). Sea fog detection using U-Net deep learning model based on MODIS data. Proceedings of the 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Amsterdam, The Netherlands. 5. Cloud Image Retrieval for Sea Fog Recognition (CIR-SFR) Using Double Branch Residual Neural Network;Hu;IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.,2023
|
|