Feature-Decision Level Collaborative Fusion Network for Hyperspectral and LiDAR Classification

Abstract

The fusion-based classification of hyperspectral (HS) and light detection and ranging (LiDAR) images has become a prominent research topic, as their complementary information can effectively improve classification performance. The current methods encompass pixel-, feature- and decision-level fusion. Among them, feature- and decision-level fusion have emerged as the mainstream approaches. Collaborative fusion of these two levels can enhance classification accuracy. Although various methods have been proposed, some shortcomings still exist. On one hand, current methods ignore the shared advanced features between HS and LiDAR images, impeding the integration of multimodal features and thereby limiting the classification performance. On the other hand, the existing methods face difficulties in achieving a balance between feature- and decision-level contributions, or they simply overlook the significance of one level and fail to utilize it effectively. In this paper, we propose a novel feature-decision level collaborative fusion network (FDCFNet) for hyperspectral and LiDAR classification to alleviate these problems. Specifically, a multilevel interactive fusion module is proposed to indirectly connect hyperspectral and LiDAR flows to refine the spectral-elevation information. Moreover, the fusion features of the intermediate branch can further enhance the shared-complementary information of hyperspectral and LiDAR to reduce the modality differences. In addition, a dynamic weight selection strategy is meticulously designed to adaptively assign weight to the output of three branches at the decision level. Experiments on three public benchmark datasets demonstrate the effectiveness of the proposed methods.