Online assembly inspection integrating lightweight hybrid neural network with positioning box matching

Abstract

Abstract Assembly inspection methods have been widely used in the process of mechanical product assembly for quality issues. However, some challenges remain to be solved, such as low detection efficiency, poor accuracy and sensitive to camera view. This paper proposes an online assembly inspection scheme based on lightweight hybrid neural network and positioning box matching. A lightweight hybrid neural network is proposed to simultaneously detect key points and parts with high accuracy and strong robustness. Utilizing the key points detection results, the transformation relationships between the in-site assembly images and the standard templates are solved. According to the results of assembly parts detection, the detected 2D positioning bounding boxes are matched with those in the standard assembly templates, so as to evaluate whether the current step has quality problems. In addition, the proposed method is tested on an assembly dataset constructed in this paper. For key points detection, the average error is less than1 pixel. For parts detection, the mean average precision is 97.66%. The missing and wrong assembly inspection results show that the average F1score reaches 93.96%. This inspection method can be employed to detect the missing and wrong assembly errors of each assembly step online, improving the assembly quality of products.