Metrics for Assessing Generalization of Deep Reinforcement Learning in Parameterized Environments-Reference-Cited by-同舟云学术

Metrics for Assessing Generalization of Deep Reinforcement Learning in Parameterized Environments

Published:2023-12-01 Issue:1 Volume:14 Page:45-61
ISSN:2449-6499
Container-title:Journal of Artificial Intelligence and Soft Computing Research
language:en
Short-container-title:

Author:

Aleksandrowicz Maciej¹^ORCID,Jaworek-Korjakowska Joanna¹^ORCID

Affiliation:

1. 1 Department of Automatic Control and Robotics , AGH University of Krakow , al. A. Mickiewicza 30, Building B-1 , Kraków

Abstract

Abstract In this work, a study focusing on proposing generalization metrics for Deep Reinforcement Learning (DRL) algorithms was performed. The experiments were conducted in DeepMind Control (DMC) benchmark suite with parameterized environments. The performance of three DRL algorithms in selected ten tasks from the DMC suite has been analysed with existing generalization gap formalism and the proposed ratio and decibel metrics. The results were presented with the proposed methods: average transfer metric and plot for environment normal distribution. These efforts allowed to highlight major changes in the model’s performance and add more insights about making decisions regarding models’ requirements.

Publisher

Walter de Gruyter GmbH

Subject

Artificial Intelligence,Computer Vision and Pattern Recognition,Hardware and Architecture,Modeling and Simulation,Information Systems

Link

https://www.sciendo.com/pdf/10.2478/jaiscr-2024-0003

Reference33 articles.

1. Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rockt¨aschel. A Survey of Zero-shot Generalisation in Deep Reinforcement Learning. Journal of Artificial Intelligence Research, 76: 201–264, January 2023. ISSN 1076-9757. doi:10.1613/jair.1.14174.

2. Katsuhiko Ogata. Modern Control Engineering. Prentice Hall, 2010. ISBN 978-0-13-615673-4.

3. Richard S. Sutton and Andrew G. Barto. Sutton & Barto Book: Reinforcement Learning: An Introduction. 2018. ISBN 978-0-262-03924-6.

4. Dimitri P. Bertsekas. Reinforcement Learning and Optimal Control. 2019. ISBN 978-1-886529-39-7.

5. Hiroki Furuta and et al. Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning. In Proceedings of the 38th International Conference on Machine Learning, pages 3541–3552. PMLR, July 2021.