Human Body Shape Reconstruction from Binary Image Using Convolutional Neural Network

Abstract

Abstract Human body modeling is an important part of virtual try-on. In order to quickly reconstruct the three-dimensional(3D) human body based on the minimum input, we propose a new method for accurate reconstruction of the human body by inputting binary images from either single view or multiple views. We first encode the shape of the human body via Principal Component Analysis (PCA) to extract the low dimensional shape descriptor. Secondly, we design a novel Body Reconstruction Convolutional Neural Network (BRCNN) with two branches, which could capture deep correlated features from different views and merge them. Given the obtained statistical shape space of the human body, we jointly train the BRCNN to learn a global mapping from the input to the shape descriptor which can be then decoded to points cloud for the reconstruction of various body shapes under neutral poses. The experimental results show that compared with the existing human reconstruction technology, the accuracy has been improved by 1.07%, and the prediction results of the two views are better than those from the single view. Further investigation also reveals that the prediction results of the weight-sharing network are better than the network without weight-sharing.