Abstract
With the continuous development of deep reinforcement learning in intelligent control, combining automatic curriculum learning and deep reinforcement learning can improve the training performance and efficiency of algorithms from easy to difficult. Most existing automatic curriculum learning algorithms perform curriculum ranking through expert experience and a single network, which has the problems of difficult curriculum task ranking and slow convergence speed. In this paper, we propose a curriculum reinforcement learning method based on K-Fold Cross Validation that can estimate the relativity score of task curriculum difficulty. Drawing lessons from the human concept of curriculum learning from easy to difficult, this method divides automatic curriculum learning into a curriculum difficulty assessment stage and a curriculum sorting stage. Through parallel training of the teacher model and cross-evaluation of task sample difficulty, the method can better sequence curriculum learning tasks. Finally, simulation comparison experiments were carried out in two types of multi-agent experimental environments. The experimental results show that the automatic curriculum learning method based on K-Fold cross-validation can improve the training speed of the MADDPG algorithm, and at the same time has a certain generality for multi-agent deep reinforcement learning algorithm based on the replay buffer mechanism.
Funder
National Natural Science Foundation of China
National Defense Scientific Research Program
Subject
General Physics and Astronomy
Reference30 articles.
1. Foglino, F., Christakou, C.C., and Gutierrez, R.L. (2019). Curriculum learning for cumulative return maximization. arXiv.
2. Mnih, V., Kavukcuoglu, K., and Silver, D. (2013). Playing atari with deep reinforcement learning. arXiv.
3. Curriculum-guided hindsight experience replay;Fang;Adv. Neural Inf. Process. Syst.,2019
4. Mastering the game of Go with deep neural networks and tree search;Silver;Nature,2016
5. Palmer, G., Tuyls, K., and Bloembergen, D. (2017). Lenient multi-agent deep reinforcement learning. arXiv.
Cited by
6 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献