Mineral Photos Recognition Based on Feature Fusion and Online Hard Sample Mining

Abstract

Mineral recognition is of importance in geological research. Traditional mineral recognition methods need professional knowledge or special equipment, are susceptible to human experience, and are inconvenient to carry in some conditions such as in the wild. The development of computer vision provides a possibility for convenient, fast, and intelligent mineral recognition. Recently, several mineral recognition methods based on images using a neural network have been proposed for this aim. However, these methods do not exploit features extracted from the backbone network or available information of the samples in the mineral dataset sufficiently, resulting in low recognition accuracy. In this paper, a method based on feature fusion and online hard sample mining is proposed to improve recognition accuracy by using only mineral photo images. This method first fuses multi-resolution features extracted from ResNet-50 to obtain comprehensive information of mineral photos, and then proposes the weighted top-k loss to emphasize the learning of hard samples. Based on a dataset consisting of 14,986 images of 22 common minerals, the proposed method with 10-fold cross-validation achieves a Top1 accuracy of 88.01% on the validation image set, surpassing those of Inception-v3 and EfficientNet-B0 by a margin of 1.88% and 1.29%, respectively, which demonstrates the good prospect of the proposed method for convenient and reliable mineral recognition using mineral photos only.