Deep and shallow feature fusion framework for remote sensing open pit coal mine scene recognition

Abstract

Abstract Grasping the situation of land use and damage in the open-pit coal mining area is of great significance to the scientific supervision and management of the area. In the existing recognition methods, the traditional features rely on manual design and the ability to express features is weak, while the deep learning methods rely too much on samples. In order to overcome the above limitations, this paper proposes a three-branch feature extraction framework that fuses deep features (DF) and shallow features (SF). Deep features mainly include two modules: key feature extraction module and contextual feature extraction module. The key feature extraction module consists of multi-level feature extraction and an attention mechanism that highlights shallow information. The new attention mechanism captures the relationship between neighboring features and adds key information from the highlighted shallow features to the final feature layer. The contextual feature extraction module introduces the Graph Convolutional Network (GCN) model to effectively reveal the correlation between the local information of the scene to obtain finer features. The shallow features are extracted by Gray-Level Co-occurrence Matrix (GLCM) to characterize the local variations of the texture, and Gabor to characterize the overall texture variations. The two kinds of features are fused and input into the particle swarm algorithm optimized support vector machine (PSO-SVM) for scene classification and recognition of remote sensing(RS) images. The method was experimented on the AID dataset and RSSCN7 dataset and the experimental results showed that the method outperforms other models.