An adaptive weighting multimodal fusion classification system for steel plate surface defect

Abstract

In steel surface inspection, an accurate steel surface defect identification method is needed to evaluate the impact of defects on structural performance and system maintenance. Traditionally, the recognition accuracy of methods based on handcrafted features is limited, but the system performance can be improved by feature fusion extracted by different methods. Therefore, this research uses the pre-trained convolutional neural network (CNN) combined with transfer learning to extract effective abstract features, and carries out adaptive weighting multimodal fusion of three the abstract features and handcrafted feature sets at the decision-making level, that is, proposes an adaptive weighting multimodal fusion classification system. The system uses handcrafted features as a supplement to abstract features, and accurately classifies steel surface defects in completely different feature representation spaces. Based on the NEU steel plate surface defect benchmark database, the classification results of feature sets before and after fusion are compared and analyzed. The experimental results show that the classification accuracy of the fusion system is improved by at least 3.4% compared with that before fusion, and the final accuracy rate is 99.0%, which proves the effectiveness of the proposed system.