Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection
-
Published:2023-03-17
Issue:6
Volume:13
Page:3838
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Pei Mingjing1234ORCID, Liu Ningzhong134ORCID, Gao Pan134, Sun Han134ORCID
Affiliation:
1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China 2. School of Electronics and Information Engineering, West Anhui University, Lu’an 237012, China 3. MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing 211106, China 4. Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 210093, China
Abstract
Industrial defect detection plays an important role in smart manufacturing and is widely used in various scenarios such as smart inspection and product quality control. Currently, although utilizing a framework for knowledge distillation to identify industrial defects has achieved great progress, it is still a significant challenge task to extract better image features and prevent overfitting for student networks. In this study, a reverse knowledge distillation framework with two teachers is designed. First, for the teacher network, two teachers with different architectures are used to extract the diverse features of the images from multiple models. Second, considering the different contributions of channels and different teacher networks, the attention mechanism and iterative attention feature fusion idea are introduced. Finally, to prevent overfitting, the student network is designed with a network architecture that is inconsistent with the teacher network. Extensive experiments were conducted on Mvtec and BTAD datasets, which are industrial defect detection datasets. On the Mvtec dataset, the average accuracy values of image-level and pixel-level ROC achieved 99.43% and 97.87%, respectively. On the BTAD dataset, the average accuracy values of image-level and pixel-level ROC reached 94% and 98%, respectively. The performance on both datasets is significantly improved, demonstrating the effectiveness of our method.
Funder
Natural Science Foundation of Jiangsu Province of China Guangxi Science and Technology Project National Natural Science Foundation of China Natural Science Key Project of Anhui Provincial Education Department
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference50 articles.
1. Deep learning in computer vision: A critical review of emerging techniques and application scenarios;Chai;Mach. Learn. Appl.,2021 2. Chen, Z., Lu, Z., Gao, H., Zhang, Y., Zhao, J., Hong, D., and Zhang, B. (2022). Global to Local: A Hierarchical Detection Algorithm for Hyperspectral Image Target Detection. IEEE Trans. Geosci. Remote Sens., 60. 3. Neubauer, K., Bullard, E., and Blunt, R. (2021). Collection of Data with Unmanned Aerial Systems (UAS) for Bridge Inspection and Construction Inspection, Technical Report. 4. Kuric, I., Klarák, J., Bulej, V., Sága, M., Kandera, M., Hajdučík, A., and Tucki, K. (2022). Approach to Automated Visual Inspection of Objects Based on Artificial Intelligence. Appl. Sci., 12. 5. An efficient dimension reduction based fusion of CNN and SVM model for detection of abnormal incident in video surveillance;Sharma;J. Soft Comput. Paradig. (JSCP),2021
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|