DSSFN: A Dual-Stream Self-Attention Fusion Network for Effective Hyperspectral Image Classification

Abstract

Hyperspectral images possess a continuous and analogous spectral nature, enabling the classification of distinctive information by analyzing the subtle variations between adjacent spectra. Meanwhile, a hyperspectral dataset includes redundant and noisy information in addition to larger dimensions, which is the primary barrier preventing its use for land cover categorization. Despite the excellent feature extraction capability exhibited by convolutional neural networks, its efficacy is restricted by the constrained receptive field and the inability to acquire long-range features due to the limited size of the convolutional kernels. We construct a dual-stream self-attention fusion network (DSSFN) that combines spectral and spatial information in order to achieve the deep mining of global information via a self-attention mechanism. In addition, dimensionality reduction is required to reduce redundant data and eliminate noisy bands, hence enhancing the performance of hyperspectral classification. A unique band selection algorithm is proposed in this study. This algorithm, which is based on a sliding window grouped normalized matching filter for nearby bands (SWGMF), can minimize the dimensionality of the data while preserving the corresponding spectral information. Comprehensive experiments are carried out on four well-known hyperspectral datasets, where the proposed DSSFN achieves higher classification results in terms of overall accuracy (OA), average accuracy (AA), and kappa than previous approaches. A variety of trials verify the superiority and huge potential of DSSFN.