Deep learning-based end-to-end 3D depth recovery from a single-frame fringe pattern with the MSUNet++ network

Abstract

Deep learning (DL)-based single-frame fringe pattern to 3D depth reconstruction methods have aroused extensive research interest. The goal is to estimate high-precision 3D shape from a single frame of fringe pattern with limited information. Therefore, the purpose of this work attempts to propose an end-to-end DL-based 3D reconstruction method from the single fringe pattern with excellent capability of achieving high accuracy depth recovery and geometry details preservation of tested objects. We construct a multi-scale feature fusion convolutional neural network (CNN) called MSUNet++, which incorporates discrete wavelet transform (DWT) in data preprocessing for extracting high-frequency signals of fringe patterns as input of the network. Additionally, a loss function that combines structural similarity with edge perception is established. Through these measures, high-frequency geometry details of the reconstruction results can be obviously enhanced, while the geometric shape can be effectively maintained. Ablation experiments are involved in validating the effectiveness of our proposed solution. 3D reconstructed results and analysis of generalization experiments on different tested samples imply that the proposed method in this research enjoys capabilities of higher accuracy, better detail preservation, and robustness in comparison with the compared methods.