A Multi-Scale Pseudo-Siamese Network with an Attention Mechanism for Classification of Hyperspectral and LiDAR Data

Abstract

For the remote sensing classification task, the ability of a single data source to identify the ground objects remains limited due to the lack of feature diversity. As the typical remote sensing data sources, hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data can provide complementary spectral features and elevation information, respectively. To enhance classification ability, a multi-scale Pseudo-Siamese Network with attention mechanism (MA-PSNet) is proposed by fusing HSI and LiDAR data. In the network, two sub-branch networks are designed for extracting the features from HSI and LiDAR, respectively, and the connection is further established between these two branches. Specifically, a multi-scale feature learning module is incorporated, enabling the image features to be fully extracted at different scales. Similarly, a convolutional attention module is also embedded to highlight the saliency information of the objects, which makes the network training can be more targeted, thereby eventually improving the model performance for classification. The evaluation experiments of the proposed model are carried out on an urban dataset from Houston, USA, and a rural dataset from Trento, Italy. The overall accuracy (OA) of the model can reach 95.03% on the Houston data and 99.16% on the Trento data. The experimental results fully demonstrate that the proposed model has competitive performance compared with several state-of-the-art methods.