Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity

Author:

Allard Maxime1ORCID,Smith Simón C.1ORCID,Chatzilygeroudis Konstantinos2ORCID,Lim Bryan1ORCID,Cully Antoine1ORCID

Affiliation:

1. Imperial College London, UK

2. University of Patras, Greece

Abstract

In real-world environments, robots need to be resilient to damages and robust to unforeseen scenarios. Quality-Diversity (QD) algorithms have been successfully used to make robots adapt to damages in seconds by leveraging a diverse set of learned skills. A high diversity of skills increases the chances of a robot to succeed at overcoming new situations since there are more potential alternatives to solve a new task. However, finding and storing a large behavioural diversity of multiple skills often leads to an increase in computational complexity. Furthermore, robot planning in a large skill space is an additional challenge that arises with an increased number of skills. Hierarchical structures can help to reduce this search and storage complexity by breaking down skills into primitive skills. In this article, we extend the analysis of the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot adapt quickly in the physical world. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable. Experiments with a hexapod robot both in simulation and the physical world show that our method solves a maze navigation task with up to, respectively, 20% and 43% less actions than the best baselines while having 78% less complete failures.

Funder

Engineering and Physical Sciences Research Council

Publisher

Association for Computing Machinery (ACM)

Subject

Process Chemistry and Technology,Economic Geology,Fuel Technology

Reference57 articles.

1. OpenAI Ilge Akkaya Marcin Andrychowicz Maciek Chociej Mateusz Litwin Bob McGrew Arthur Petron Alex Paino Matthias Plappert Glenn Powell Raphael Ribas Jonas Schneider Nikolas Tezak Jerry Tworek Peter Welinder Lilian Weng Qiming Yuan Wojciech Zaremba and Lei Zhang. 2019. Solving Rubik’s Cube with a Robot Hand. CoRR abs/1910.07113 (2019). arXiv:1910.07113 http://arxiv.org/abs/1910.07113.

2. Maxime Allard, Simón C. Smith, Konstantinos Chatzilygeroudis, and Antoine Cully. 2022. Hierarchical quality-diversity for online damage recovery. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, New York, NY, 58–67. DOI:10.1145/3512290.3528751

3. Karl J. Åström and Björn Wittenmark. 2013. Adaptive Control. Courier Corporation.

4. David M. Bossens and Danesh Tarapore. 2021. Rapidly adapting robot swarms with swarm Map-based bayesian optimisation. In 2021 IEEE International Conference on Robotics and Automation (ICRA) . 9848–9854. 10.1109/ICRA48506.2021.9560958

5. A Self-Tuning Controller with a PID Structure

Cited by 7 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. Automating Robot Design with Multi-Level Evolution;2024 IEEE Congress on Evolutionary Computation (CEC);2024-06-30

2. Body and Brain Quality-Diversity in Robot Swarms;ACM Transactions on Evolutionary Learning and Optimization;2024-05-10

3. Evolutionary Reinforcement Learning: Hybrid Approach for Safety-Informed Fault-Tolerant Flight Control;Journal of Guidance, Control, and Dynamics;2024-05

4. Evolutionary Reinforcement Learning: A Hybrid Approach for Safety-informed Intelligent Fault-tolerant Flight Control;AIAA SCITECH 2024 Forum;2024-01-04

5. Adaptive Control Strategy for Quadruped Robots in Actuator Degradation Scenarios;The Fifth International Conference on Distributed Artificial Intelligence;2023-11-30

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3