AdaptNet: Policy Adaptation for Physics-Based Character Control-Reference-Cited by-同舟云学术

AdaptNet: Policy Adaptation for Physics-Based Character Control

Published:2023-12-05 Issue:6 Volume:42 Page:1-17
ISSN:0730-0301
Container-title:ACM Transactions on Graphics
language:en
Short-container-title:ACM Trans. Graph.

Author:

Xu Pei¹^ORCID,Xie Kaixiang²^ORCID,Andrews Sheldon³^ORCID,Kry Paul G.²^ORCID,Neff Michael⁴^ORCID,Mcguire Morgan⁵^ORCID,Karamouzas Ioannis⁶^ORCID,Zordan Victor⁷^ORCID

Affiliation:

1. Clemson University, USA and Roblox, USA

2. McGill University, Canada

3. École de Technologie Supérieure, Canada and Roblox, USA

4. University of California, Davis, USA

5. Roblox, USA and University of Waterloo, Canada

6. University of California, Riverside, USA

7. Roblox, USA and Clemson University, USA

Abstract

Motivated by humans' ability to adapt skills in the learning of new ones, this paper presents AdaptNet, an approach for modifying the latent space of existing policies to allow new behaviors to be quickly learned from like tasks in comparison to learning from scratch. Building on top of a given reinforcement learning controller, AdaptNet uses a two-tier hierarchy that augments the original state embedding to support modest changes in a behavior and further modifies the policy network layers to make more substantive changes. The technique is shown to be effective for adapting existing physics-based controllers to a wide range of new styles for locomotion, new task targets, changes in character morphology and extensive changes in environment. Furthermore, it exhibits significant increase in learning efficiency, as indicated by greatly reduced training times when compared to training from scratch or using other approaches that modify existing policies. Code is available at https://motion-lab.github.io/AdaptNet .

Funder

NSERC

NSF

Publisher

Association for Computing Machinery (ACM)

Subject

Computer Graphics and Computer-Aided Design

Link

https://dl.acm.org/doi/pdf/10.1145/3618375

Reference79 articles.

1. R. Abdal , Y. Qin , and P. Wonka . 2019. Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? . In Proc. of the IEEE/CVF Int. Conf. on Computer Vision. 4432--4441 . R. Abdal, Y. Qin, and P. Wonka. 2019. Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?. In Proc. of the IEEE/CVF Int. Conf. on Computer Vision. 4432--4441.

2. Unpaired motion style transfer from video to animation

3. A. Aghajanyan S. Gupta and L. Zettlemoyer. 2021. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 7319--7328. A. Aghajanyan S. Gupta and L. Zettlemoyer. 2021. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 7319--7328.

4. F. Alet , T. Lozano-Perez , and L. P. Kaelbling . 2018. Modular meta-learning . In Conf. on Robot Learning (Proc. of Machine Learning Research , Vol. 87). 856-- 868 . F. Alet, T. Lozano-Perez, and L. P. Kaelbling. 2018. Modular meta-learning. In Conf. on Robot Learning (Proc. of Machine Learning Research, Vol. 87). 856--868.

5. M. Andrychowicz M. Denil S. G. Colmenarejo M. W. Hoffman D. Pfau T. Schaul B. Shillingford and N. de Freitas. 2016. Learning to Learn by Gradient Descent by Gradient Descent. In Neural Information Processing Systems. 3988--3996. M. Andrychowicz M. Denil S. G. Colmenarejo M. W. Hoffman D. Pfau T. Schaul B. Shillingford and N. de Freitas. 2016. Learning to Learn by Gradient Descent by Gradient Descent. In Neural Information Processing Systems. 3988--3996.