Real-Time Segmentation of Artificial Targets Using a Dual-Modal Efficient Attention Fusion Network

Abstract

The fusion of spectral–polarimetric information can improve the autonomous reconnaissance capability of unmanned aerial vehicles (UAVs) in detecting artificial targets. However, the current spectral and polarization imaging systems typically suffer from low image sampling resolution, which can lead to the loss of target information. Most existing segmentation algorithms neglect the similarities and differences between multimodal features, resulting in reduced accuracy and robustness of the algorithms. To address these challenges, a real-time spectral–polarimetric segmentation algorithm for artificial targets based on an efficient attention fusion network, called ESPFNet (efficient spectral–polarimetric fusion network) is proposed. The network employs a coordination attention bimodal fusion (CABF) module and a complex atrous spatial pyramid pooling (CASPP) module to fuse and enhance low-level and high-level features at different scales from the spectral feature images and the polarization encoded images, effectively achieving the segmentation of artificial targets. Additionally, the introduction of the residual dense block (RDB) module refines feature extraction, further enhancing the network’s ability to classify pixels. In order to test the algorithm’s performance, a spectral–polarimetric image dataset of artificial targets, named SPIAO (spectral–polarimetric image of artificial objects) is constructed, which contains various camouflaged nets and camouflaged plates with different properties. The experimental results on the SPIAO dataset demonstrate that the proposed method accurately detects the artificial targets, achieving a mean intersection-over-union (MIoU) of 80.4%, a mean pixel accuracy (MPA) of 88.1%, and a detection rate of 27.5 frames per second, meeting the real-time requirement. The research has the potential to provide a new multimodal detection technique for enabling autonomous reconnaissance by UAVs in complex scenes.