Deep Multi-Order Spatial–Spectral Residual Feature Extractor for Weak Information Mining in Remote Sensing Imagery

Abstract

Remote sensing images (RSIs) are widely used in various fields due to their versatility, accuracy, and capacity for earth observation. Direct application of RSIs to harvest optimal results is generally difficult, especially for weak information features in the images. Thus, extracting the weak information in RSIs is reasonable to promote further applications. However, the current techniques for weak information extraction mainly focus on spectral features in hyperspectral images (HSIs), and a universal weak information extraction technology for RSI is lacking. Therefore, this study focused on mining the weak information from RSIs and proposed the deep multi-order spatial–spectral residual feature extractor (DMSRE). The DMSRE considers the global information and three-dimensional cube structures by combining low-rank representation, high-order residual quantization, and multi-granularity spectral segmentation theories. This extractor obtains spatial–spectral features from two derived sequences (deep spatial–spectral residual feature (DMSR) and deep spatial–spectral coding feature (DMSC)), and three RSI datasets (i.e., Chikusei, ZY1-02D, and Pasture datasets) were employed to validate the DMSRE method. Comparative results of the weak information extraction-based classifications (including DMSR and DMSC) and the raw image-based classifications showed the following: (i) the DMSRs can improve the classification accuracy of individual classes in fine classification applications (e.g., Asphalt class in the Chikusei dataset, from 89.12% to 95.99%); (ii) the DMSC improved the overall accuracy in rough classification applications (from 92.07% to 92.78%); and (iii) the DMSC improved the overall accuracy in RGB classification applications (from 63.25% to 63.6%), whereas DMSR improved the classification accuracy of individual classes on the RGB image (e.g., Plantain classes in the Pasture dataset, from 32.49% to 39.86%). This study demonstrates the practicality and capability of the DMSRE method to promote target recognition on RSIs and presents an alternative technique for weak information mining on RSIs, indicating the potential to extend weak information-based applications of RSIs.