Comprehensive Ocean Information-Enabled AUV Motion Planning Based on Reinforcement Learning

Abstract

Motion planning based on the reinforcement learning algorithms of the autonomous underwater vehicle (AUV) has shown great potential. Motion planning algorithms are primarily utilized for path planning and trajectory-tracking. However, prior studies have been confronted with some limitations. The time-varying ocean current affects algorithmic sampling and AUV motion and then leads to an overestimation error during path planning. In addition, the ocean current makes it easy to fall into local optima during trajectory planning. To address these problems, this paper presents a reinforcement learning-based motion planning algorithm with comprehensive ocean information (RLBMPA-COI). First, we introduce real ocean data to construct a time-varying ocean current motion model. Then, comprehensive ocean information and AUV motion position are introduced, and the objective function is optimized in the state-action value network to reduce overestimation errors. Finally, state transfer and reward functions are designed based on real ocean current data to achieve multi-objective path planning and adaptive event triggering in trajectorytracking to improve robustness and adaptability. The numerical simulation results show that the proposed algorithm has a better path planning ability and a more robust trajectory-tracking effect than those of traditional reinforcement learning algorithms.