Affiliation:
1. Tencent AI Lab, Shenzhen, Guangdong, China
2. Tencent NExT Studios, Shanghai, China
Abstract
We present a fully automatic system that can produce high-fidelity, photo-realistic three-dimensional (3D) digital human heads with a consumer RGB-D selfie camera. The system only needs the user to take a short selfie RGB-D video while rotating his/her head and can produce a high-quality head reconstruction in less than 30 s. Our main contribution is a new facial geometry modeling and reflectance synthesis procedure that significantly improves the state of the art. Specifically, given the input video a two-stage frame selection procedure is first employed to select a few high-quality frames for reconstruction. Then a differentiable renderer-based 3D Morphable Model (3DMM) fitting algorithm is applied to recover facial geometries from multiview RGB-D data, which takes advantages of a powerful 3DMM basis constructed with extensive data generation and perturbation. Our 3DMM has much larger expressive capacities than conventional 3DMM, allowing us to recover more accurate facial geometry using merely linear basis. For reflectance synthesis, we present a hybrid approach that combines parametric fitting and
Convolutional Neural Networks (CNNs)
to synthesize high-resolution albedo/normal maps with realistic hair/pore/wrinkle details. Results show that our system can produce faithful 3D digital human faces with extremely realistic details. The main code and the newly constructed 3DMM basis is publicly available.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Graphics and Computer-Aided Design
Reference81 articles.
1. Least-squares fitting of two 3-D point sets;Arun K. Somani;IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9,1987
2. High-quality single-shot capture of facial geometry;Beeler Thabo;ACM Trans. Graph. (Proc. SIGGRAPH),2010
3. Lightweight eye capture using a parametric model;Bérard Pascal;ACM Trans. Graph. (Proc. SIGGRAPH),2016
Cited by
34 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献