Transformer-Based Hierarchical Multiscale Feature Fusion Internal Wave Detection and Dataset

Abstract

Ocean internal waves (IWs) are widespread submesoscale dynamical phenomena in oceans, and they have important impacts on energy transfer, nutrient transport, and human activities. In this study, Sentinel-1 synthetic aperture radar (SAR) images from 2014 to 2023 were collected to construct a global IW dataset (S1-IW-2023) through a series of optimized data processing. S1-IW-2023 was constructed to address the issue of insufficient data and lack of variation in the deep learning IW datasets; it can be used in IW studies using deep learning methods to enhance model generalizability and robustness. Moreover, considering the limitations of existing convolutional neural network (CNN)-based IW detection models in handling complex interferences in SAR images, resulting in frequent false positives, false negatives, or inaccurate bounding box positioning, we employed transfer learning to train a Transformer-based hierarchical IW detector, IWD-Net, that extracts features via Swin Transformer and fuses the visual, semantic, and contextual features of IWs via a multiscale feature fusion network. Experimental results demonstrated the effectiveness of applying Transformer concepts to IW detection for addressing complex interferences. This study provides an efficient and stable new method for extracting IW features from massive SAR data and lays the foundation for applying Transformer concepts to detect IWs in SAR images.