A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model

Abstract

Aiming at the demand for extracting the three-dimensional shapes of droplets in microelectronic packaging, life science, and some related fields, as well as the problems of complex calculation and slow running speed of conventional shape from shading (SFS) illumination reflection models, this paper proposes a Lambert–Phong hybrid model algorithm to recover the 3D shapes of micro-droplets based on the mask regions with convolutional neural network features (R-CNN) method to extract the highlight region of the droplet surface. This method fully integrates the advantages of the Lambertian model’s fast running speed and the Phong model’s high accuracy for reconstruction of the highlight region. First, the Mask R-CNN network is used to realize the segmentation of the highlight region of the droplet and obtain its coordinate information. Then, different reflection models are constructed for the different reflection regions of the droplet, and the Taylor expansion and Newton iteration method are used for the reflection model to get the final height of all positions. Finally, a three-dimensional reconstruction experimental platform is built to analyze the accuracy and speed of the algorithm on the synthesized hemisphere image and the actual droplet image. The experimental results show that the proposed algorithm based on mask R-CNN had better precision and shorter running time. Hence, this paper provides a new approach for real-time measurement of 3D droplet shape in the dispensing state.