Multi‐Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL Constraints

Abstract

AbstractAiming at enhancing the rationality and robustness of the results of single‐view image‐based human reconstruction and acquiring richer surface details, we propose a multi‐level reconstruction framework based on implicit functions. This framework first utilizes the predicted SMPL model (Skinned Multi‐Person Linear Model) as a prior to further predict consistent 2.5D sketches (depth map and normal map), and then obtains a coarse reconstruction result through an Implicit Function fitting network (IF‐Net). Subsequently, with a pixel‐aligned feature extraction module and a fine IF‐Net, the strong constraints imposed by SMPL are relaxed to add more surface details to the reconstruction result and remove noise. Finally, to address the trade‐off between surface details and rationality under complex poses, we propose a novel fusion repair algorithm that reuses existing information. This algorithm compensates for the missing parts of the fine reconstruction results with the coarse reconstruction results, leading to a robust, rational, and richly detailed reconstruction. The final experiments prove the effectiveness of our method and demonstrate that it achieves the richest surface details while ensuring rationality. The project website can be found at https://github.com/MXKKK/2.5D‐MLIF.