A causal convolutional neural network for multi-subject motion modeling and generation
-
Published:2023-11-30
Issue:1
Volume:10
Page:45-59
-
ISSN:2096-0433
-
Container-title:Computational Visual Media
-
language:en
-
Short-container-title:Comp. Visual Media
Author:
Hou Shuaiying,Wang Congyi,Zhuang Wenlin,Chen Yu,Wang Yangang,Bao Hujun,Chai Jinxiang,Xu Weiwei
Abstract
AbstractInspired by the success of WaveNet in multi-subject speech synthesis, we propose a novel neural network based on causal convolutions for multi-subject motion modeling and generation. The network can capture the intrinsic characteristics of the motion of different subjects, such as the influence of skeleton scale variation on motion style. Moreover, after fine-tuning the network using a small motion dataset for a novel skeleton that is not included in the training dataset, it is able to synthesize high-quality motions with a personalized style for the novel skeleton. The experimental results demonstrate that our network can model the intrinsic characteristics of motions well and can be applied to various motion modeling and synthesis tasks.
Publisher
Springer Science and Business Media LLC
Subject
Artificial Intelligence,Computer Graphics and Computer-Aided Design,Computer Vision and Pattern Recognition
Reference45 articles.
1. Zhang, P. F.; Lan, C. L.; Zeng, W. J.; Xing, J. L.; Xue, J. R.; Zheng, N. N. Semantics-guided neural networks for efficient skeleton-based human action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1109–1118, 2020. 2. Chen, Z.; Li, S. C.; Yang, B.; Li, Q. H.; Liu, H. Multi-scale spatial temporal graph convolutional network for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35, No. 2, 1113–1122, 2021. 3. Gui, L. Y.; Wang, Y. X.; Liang, X. D.; Moura, J. M. F. Adversarial geometry-aware human motion prediction. In: Computer Vision–ECCV 2018. Lecture Notes in Computer Science, Vol. 11208. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 823–842, 2018. 4. Wang, T. C.; Liu, M. Y.; Tao, A.; Liu, G. L.; Kautz, J.; Catanzaro, B. Few-shot video-to-video synthesis. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article No. 451, 5013–5024, 2019. 5. Taylor, G. W.; Hinton, G. E. Factored conditional restricted Boltzmann Machines for modeling motion style. In: Proceedings of the 26th Annual International Conference on Machine Learning, 1025–1032, 2009.
|
|