Affiliation:
1. Carnegie Mellon University and Meta Reality Labs Research
2. Meta Reality Labs Research
3. University of Minnesota
4. Carnegie Mellon University and Meta AI
Abstract
Despite recent progress in developing animatable full-body avatars, realistic modeling of clothing - one of the core aspects of human self-expression - remains an open challenge. State-of-the-art physical simulation methods can generate realistically behaving clothing geometry at interactive rates. Modeling photorealistic appearance, however, usually requires physically-based rendering which is too expensive for interactive applications. On the other hand, data-driven deep appearance models are capable of efficiently producing realistic appearance, but struggle at synthesizing geometry of highly dynamic clothing and handling challenging body-clothing configurations. To this end, we introduce pose-driven avatars with explicit modeling of clothing that exhibit both photorealistic appearance learned from real-world data and realistic clothing dynamics. The key idea is to introduce a neural clothing appearance model that operates on top of explicit geometry: at training time we use high-fidelity tracking, whereas at animation time we rely on physically simulated geometry. Our core contribution is a physically-inspired appearance network, capable of generating photorealistic appearance with view-dependent and dynamic shadowing effects even for unseen body-clothing configurations. We conduct a thorough evaluation of our model and demonstrate diverse animation results on several subjects and different types of clothing. Unlike previous work on photorealistic full-body avatars, our approach can produce much richer dynamics and more realistic deformations even for many examples of loose clothing. We also demonstrate that our formulation naturally allows clothing to be used with avatars of different people while staying fully animatable, thus enabling, for the first time, photorealistic avatars with novel clothing.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Graphics and Computer-Aided Design
Reference104 articles.
1. Tex2Shape: Detailed Full Human Body Geometry From a Single Image
2. Driving-signal aware full-body avatars
3. David Baraff and Andrew Witkin . 1998 . Large steps in cloth simulation . In Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings. David Baraff and Andrew Witkin. 1998. Large steps in cloth simulation. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings.
4. PBNS
5. DeePSD: Automatic Deep Skinning And Pose Space Deformation For 3D Garment Animation
Cited by
7 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献