A Multi-Ship Collision Avoidance Algorithm Using Data-Driven Multi-Agent Deep Reinforcement Learning

Author:

Niu Yihan1ORCID,Zhu Feixiang1ORCID,Wei Moxuan1,Du Yifan1,Zhai Pengyu2

Affiliation:

1. Navigation College, Dalian Maritime University, Dalian 116026, China

2. School of Transportation and Logistics, Dalian University of Technology, Dalian 116024, China

Abstract

Maritime Autonomous Surface Ships (MASS) are becoming of interest to the maritime sector and are also on the agenda of the International Maritime Organization (IMO). With the boom in global maritime traffic, the number of ships is increasing rapidly. The use of intelligent technology to achieve autonomous collision avoidance is a hot issue widely discussed in the industry. In the endeavor to solve this problem, multi-ship coordinated collision avoidance has become a crucial challenge. This paper proposes a multi-ship autonomous collision avoidance decision-making algorithm by a data-driven method and adopts the Multi-agent Deep Reinforcement Learning (MADRL) framework for its design. Firstly, the overall framework of this paper and its components follow the principle of “reality as primary and simulation as supplementary”, so a real data-driven AIS (Automatic Identification System) dominates the model construction. Secondly, the agent’s observation state is determined by quantifying the hazardous area. Then, based on a full understanding of the International Regulations for Preventing Collisions at Sea (COLREGs) and the preliminary data collection, this paper combines the statistical results of the real water traffic data to guide and design the algorithm framework and selects the representative influencing factors to be designed in the collision avoidance decision-making algorithm’s reward function. Next, we train the algorithmic model using both real data and simulation data. Meanwhile, Prioritized Experience Replay (PER) is adopted to accelerate the model’s learning efficiency. Finally, 40 encounter scenarios are designed and extended to verify the algorithm performance based on the idea of the Imazu problem. The experimental results show that this algorithm can efficiently make a ship collision avoidance decision in compliance with COLREGs. Multi-agent learning through shared network policies can ensure that the agents pass beyond the safe distance in unknown environments. We can apply the trained model to the system with different numbers of agents to provide a reference for the research of autonomous collision avoidance in ships.

Funder

the National Key Research and Development Program of China

Publisher

MDPI AG

Subject

Ocean Engineering,Water Science and Technology,Civil and Structural Engineering

Reference34 articles.

1. European Maritime Safety Agency (2021). Annual Overview of Marine Casualties and Incidents 2021, EMSA. Available online: https://www.emsa.europa.eu/newsroom/latest-news/item/4266-annual-overview-of-marine-casualties-and-incidents-2020.html.

2. Maritime Safety Committee (2018). Report of the Maritime Safety Committee on Its Ninety-Ninth Session, IMO. Available online: https://www.imo.org/en/MediaCentre/MeetingSummaries/Pages/MSC-99th-session.aspx.

3. Wei, G., and Kuo, W. (2022). COLREGs-Compliant Multi-Ship Collision Avoidance Based on Multi-Agent Reinforcement Learning Technique. J. Mar. Sci. Eng., 10.

4. Research progress and trend of autonomous collision avoidance technology for marine ships;Zhang;J. Dalian Marit. Univ.,2022

5. Papadimitrakis, M., Stogiannos, M., Sarimveis, H., and Alexandridis, A. (2021). Multi-Ship Control and Collision Avoidance Using MPC and RBF-Based Trajectory Predictions. Sensors, 21.

Cited by 2 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3