A second-order dynamic and static ship path planning model based on reinforcement learning and heuristic search algorithms

Abstract

AbstractShip path planning plays an important role in the intelligent decision-making system which can provide important navigation information for ship and coordinate with other ships via wireless networks. However, existing methods still suffer from slow path planning and low security problems. In this paper, we propose a second-order ship path planning model, which consists of two main steps, i.e., first-order static global path planning and second-order dynamic local path planning. Specifically, we first create a raster map using ArcGIS. Second, the global path planning is performed on the raster map based on the Dyna-Sarsa($$\lambda$$ λ ) model, which integrates the eligibility trace and the Dyna framework on the Sarsa algorithm. Particularly, the eligibility trace has a short-term memory for the trajectory, which can improve the convergence speed of the model. Meanwhile, the Dyna framework obtains simulation experience through simulation training, which can further improve the convergence speed of the model. Then, the improved ship trajectory prediction model based on stacked bidirectional gated recurrent unit is used to identify the risk of ship collision and switch the path planning from the first order to the second order. Finally, the second-order dynamic local path planning is presented based on the FCC-A* algorithm, where the cost function of the traditional path planning A* algorithm is rewritten using the fuzzy collision cost membership function (fuzzy collision cost, FCC) to reduce the collision risk of ships. The proposed model is evaluated on the Baltic Sea geographic information and ship trajectory datasets. The experimental results show that the eligibility trace and the Dyna learning framework in the proposed model can effectively improve the planning efficiency of the ship’s global path planning, and the collision risk membership function can effectively reduce the number of collisions in A* local path planning and thus improve the navigation safety of encountering ships.