Affiliation:
1. Alva’s Institute of Engineering and Technology, Mijar, Karnataka, India
Abstract
This paper provides an in-depth analysis of the present state of Deep Reinforcement Learning (DRL) applications in Unmanned Underwater Vehicles (UUVs). Addressing the persistent challenges related to data inefficiency and performance degradation in physical platforms, particularly when faced with unforeseen variations, the paper introduces the innovative Biologically-Inspired Experience Replay (BIER) method. This approach incorporates two distinct memory buffers to enhance learning efficiency. The paper assesses the generalization capabilities of BIER through training neural network controllers on diverse tasks, spanning from inverted pendulum stabilization to simulating half-cheetah running. Furthermore, BIER is integrated with the Soft Actor-Critic (SAC) method for UUV stabilization under unknown environmental dynamics. Evaluation in a ROS-based UUV simulator, incorporating increasingly complex scenarios, showcases BIER's superior performance over traditional Experience Replay (ER) methods, achieving optimal UUV control in half the time. This review contributes valuable insights into the challenges and advancements in applying DRL methods to UUVs, highlighting the BIER method's promising potential to improve adaptability and efficiency in UUV manoeuvring tasks, leading to more robust and agile underwater vehicle control systems for more robust and agile underwater vehicle control systems. .
Reference13 articles.
1. Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533. DOI: 10.1038/nature14236.
2. Lillicrap, T.P., Hunt, J.J., Pritzel, A., et al. (2016). Continuous control with deep reinforcement learning. Proceedings of the International Conference on Learning Representations (ICLR).
3. Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018). Soft Actor- Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the 35th International Conference on Machine Learning (ICML).
4. Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized Experience Replay. Proceedings of the International Conference on Learning Representations (ICLR).
5. Riedmiller, M. (2005). Neural fitted Q iteration – first experiences with a data efficient neural reinforcement learning method. European Conference on Machine Learning (ECML). Springer, Berlin, Heidelberg. Fujimoto, S., van Hoof, H., & Meger, D. (2018). Addressing Function Approximation Error in Actor-Critic Methods. Proceedings of the 35th International Conference on Machine Learning (ICML).