Boosting Deep Reinforcement Learning Agents with Generative Data Augmentation
-
Published:2023-12-29
Issue:1
Volume:14
Page:330
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Papagiannis Tasos1ORCID, Alexandridis Georgios1ORCID, Stafylopatis Andreas1
Affiliation:
1. School of Electrical & Computer Engineering, National Technical University of Athens, Zografou Campus, 15780 Athens, Greece
Abstract
Data augmentation is a promising technique in improving exploration and convergence speed in deep reinforcement learning methodologies. In this work, we propose a data augmentation framework based on generative models for creating completely novel states and increasing diversity. For this purpose, a diffusion model is used to generate artificial states (learning the distribution of original, collected states), while an additional model is trained to predict the action executed between two consecutive states. These models are combined to create synthetic data for cases of high and low immediate rewards, which are encountered less frequently during the agent’s interaction with the environment. During the training process, the synthetic samples are mixed with the actually observed data in order to speed up agent learning. The proposed methodology is tested on the Atari 2600 framework, producing realistic and diverse synthetic data which improve training in most cases. Specifically, the agent is evaluated on three heterogeneous games, achieving a reward increase of up to 31%, although the results indicate performance variance among the different environments. The augmentation models are independent of the learning process and can be integrated to different algorithms, as well as different environments, with slight adaptations.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference50 articles.
1. A survey on image data augmentation for deep learning;Shorten;J. Big Data,2019 2. Taylor, L., and Nitschke, G. (2018, January 18–21). Improving Deep Learning with Generic Data Augmentation. Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, India. 3. Kim, E.K., Lee, H., Kim, J.Y., and Kim, S. (2020). Data augmentation method by applying color perturbation of inverse PSNR and geometric transformations for object recognition based on deep learning. Appl. Sci., 10. 4. Geometric transformation-based data augmentation on defect classification of segmented images of semiconductor materials using a ResNet50 convolutional neural network;Morales;Expert Syst. Appl.,2022 5. Bowles, C., Chen, L., Guerrero, R., Bentley, P., Gunn, R., Hammers, A., Dickie, D.A., Hernández, M.V., Wardlaw, J., and Rueckert, D. (2018). Gan augmentation: Augmenting training data using generative adversarial networks. arXiv.
|
|