A Novel Dynamically Adjusted Entropy Algorithm for Collision Avoidance in Autonomous Ships Based on Deep Reinforcement Learning

Abstract

Decision-making for collision avoidance in complex maritime environments is a critical technology in the field of autonomous ship navigation. However, existing collision avoidance decision algorithms still suffer from unstable strategy exploration and poor compliance with regulations. To address these issues, this paper proposes a novel autonomous ship collision avoidance algorithm, the dynamically adjusted entropy proximal policy optimization (DAE-PPO). Firstly, a reward system suitable for complex maritime encounter scenarios is established, integrating the International Regulations for Preventing Collisions at Sea (COLREGs) with collision risk assessment. Secondly, the exploration mechanism is optimized using a quadratically decreasing entropy method to effectively avoid local optima and enhance strategic performance. Finally, a simulation testing environment based on Unreal Engine 5 (UE5) was developed to conduct experiments and validate the proposed algorithm. Experimental results demonstrate that the DAE-PPO algorithm exhibits significant improvements in efficiency, success rate, and stability in collision avoidance tests. Specifically, it shows a 45% improvement in success rate per hundred collision avoidance attempts compared to the classic PPO algorithm and a reduction of 0.35 in the maximum collision risk (CR) value during individual collision avoidance tasks.