NMSCANet: stereo matching network for speckle variations in single-shot speckle projection profilometry

Abstract

In single-shot speckle projection profilometry (SSPP), the projected speckle inevitably undergoes changes in shape and size due to variations such as viewing angles, complex surface modulations of the test object and different projection ratios. These variations introduce randomness and unpredictability to the speckle features, resulting in erroneous or missing feature extraction and subsequently degrading 3D reconstruction accuracy across the tested surface. This work strives to explore the relationship between speckle size variations and feature extraction, and address the issue solely from the perspective of network design by leveraging specific variations in speckle size without expanding the training set. Based on the analysis of the relationship between speckle size variations and feature extraction, we introduce the NMSCANet, enabling the extraction of multi-scale speckle features. Multi-scale spatial attention is employed to enhance the perception of complex and varying speckle features in space, allowing comprehensive feature extraction across different scales. Channel attention is also employed to selectively highlight the most important and representative feature channels in each image, which is able to enhance the detection capability of high-frequency 3D surface profiles. Especially, a real binocular 3D measurement system and its digital twin with the same calibration parameters are established. Experimental results imply that NMSCANet can also exhibit more than 8 times the point cloud reconstruction stability (Std) on the testing set, and the smallest change range in terms of Mean~dis (0.0614 mm - 0.4066 mm) and Std (0.0768 mm - 0.7367 mm) when measuring a standard sphere and plane compared to other methods, faced with the speckle size changes, meanwhile NMSCANet boosts the disparity matching accuracy (EPE) by over 35% while reducing the matching error (N-PER) by over 62%. Ablation studies and validity experiments collectively substantiate that our proposed modules and constructed network have made significant advancements in enhancing network accuracy and robustness against speckle variations.