Affiliation:
1. Università della Svizzera italiana, Switzerland
Abstract
Deep Reinforcement Learning (DRL) has received a lot of attention from the research community in recent years. As the technology moves away from game playing to practical contexts, such as autonomous vehicles and robotics, it is crucial to evaluate the quality of DRL agents.
In this paper, we propose a search-based approach to test such agents. Our approach, implemented in a tool called
Indago
, trains a classifier on failure and non-failure environment (i.e., pass) configurations resulting from the DRL training process. The classifier is used at testing time as a surrogate model for the DRL agent execution in the environment, predicting the extent to which a given environment configuration induces a failure of the DRL agent under test. The failure prediction acts as a fitness function, guiding the generation towards failure environment configurations, while saving computation time by deferring the execution of the DRL agent in the environment to those configurations that are more likely to expose failures.
Experimental results show that our search-based approach finds 50% more failures of the DRL agent than state-of-the-art techniques. Moreover, such failures are, on average, 78% more diverse; similarly, the behaviors of the DRL agent induced by failure configurations are 74% more diverse.
Publisher
Association for Computing Machinery (ACM)
Reference76 articles.
1. Testing advanced driver assistance systems using multi-objective search and neural networks
2. Testing autonomous cars for feature interaction failures using many-objective search
3. Marcin Andrychowicz Filip Wolski Alex Ray Jonas Schneider Rachel Fong Peter Welinder Bob McGrew Josh Tobin Pieter Abbeel and Wojciech Zaremba. 2017. Hindsight experience replay. arXiv preprint arXiv:1707.01495(2017). Marcin Andrychowicz Filip Wolski Alex Ray Jonas Schneider Rachel Fong Peter Welinder Bob McGrew Josh Tobin Pieter Abbeel and Wojciech Zaremba. 2017. Hindsight experience replay. arXiv preprint arXiv:1707.01495(2017).
4. A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering
5. R. Ben Abdessalem , S. Nejati , L. C. Briand , and T. Stifter . 2018 . Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). 1016–1026 . https://doi.org/10.1145/3 1801 55.3180160 10.1145/3180155.3180160 R. Ben Abdessalem, S. Nejati, L. C. Briand, and T. Stifter. 2018. Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). 1016–1026. https://doi.org/10.1145/3180155.3180160
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Learning and Repair of Deep Reinforcement Learning Policies from Fuzz-Testing Data;Proceedings of the 46th IEEE/ACM International Conference on Software Engineering;2024-02-06