DAFNet: Generating Diverse Actions for Furniture Interaction by Learning Conditional Pose Distribution

Abstract

AbstractWe present DAFNet, a novel data‐driven framework capable of generating various actions for indoor environment interactions. By taking desired root and upper‐body poses as control inputs, DAFNet generates whole‐body poses suitable for furniture of various shapes and combinations. To enable the generation of diverse actions, we introduce an action predictor that automatically infers the probabilities of individual action types based on the control input and environment. The action predictor is learned in an unsupervised manner by training Gaussian Mixture Variational Autoencoder (GMVAE). Additionally, we propose a two‐part normalizing flow‐based pose generator that sequentially generates upper and lower body poses. This two‐part model improves motion quality and the accuracy of satisfying conditions over a single model generating the whole body. Our experiments show that DAFNet can create continuous character motion for indoor scene scenarios, and both qualitative and quantitative evaluations demonstrate the effectiveness of our framework.We propose DAFNet, a novel data‐driven framework that can generate various actions for indoor environment interactions. Given the desired root and upper‐body pose as control inputs, DAFNet generates whole‐body poses for a character appropriate for furniture of various shapes and combinations.image