Affiliation:
1. University of Wisconsin-Madison
2. University of Utah
3. Weta Digital
4. University of Wisconsin-Madison and Weta Digital
Abstract
Humans and animals can control their bodies to generate a wide range of motions via low-dimensional action signals representing high-level goals. As such, human bodies and faces are prime examples of
active
objects, which can affect their shape via an internal actuation mechanism. This paper explores the following proposition: given a training set of example poses of an active deformable object, can we learn a low-dimensional control space that could reproduce the training set and generalize to new poses? In contrast to popular machine learning methods for dimensionality reduction such as auto-encoders, we model our active objects in a physics-based way. We utilize a differentiable, quasistatic, physics-based simulation layer and combine it with a decoder-type neural network. Our differentiable physics layer naturally fits into deep learning frameworks and allows the decoder network to learn actuations that reach the desired poses after physics-based simulation. In contrast to modeling approaches where users build anatomical models from first principles, medical literature or medical imaging, we do not presume knowledge of the underlying musculature, but learn the structure and control of the actuation mechanism directly from the input data. We present a training paradigm and several scalability-oriented enhancements that allow us to train effectively while accommodating high-resolution volumetric models, with as many as a quarter million simulation elements. The prime demonstration of the efficacy of our example-driven modeling framework targets facial animation, where we train on a collection of input expressions while generalizing to unseen poses, drive detailed facial animation from sparse motion capture input, and facilitate expression sculpting via direct manipulation.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Graphics and Computer-Aided Design
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. An Implicit Physical Face Model Driven by Expression and Style;SIGGRAPH Asia 2023 Conference Papers;2023-12-10
2. MuscleVAE: Model-Based Controllers of Muscle-Actuated Characters;SIGGRAPH Asia 2023 Conference Papers;2023-12-10
3. SoftDECA: Computationally Efficient Physics-Based Facial Animations;ACM SIGGRAPH Conference on Motion, Interaction and Games;2023-11-15
4. A Generalized Constitutive Model for Versatile MPM Simulation and Inverse Learning with Differentiable Physics;Proceedings of the ACM on Computer Graphics and Interactive Techniques;2023-08-16
5. Data-Free Learning of Reduced-Order Kinematics;Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings;2023-07-23