Abstract
<div class="section abstract"><div class="htmlview paragraph">The on-ramp merging driving scenario is challenging for achieving the highest-level autonomous driving. Current research using reinforcement learning methods to address the on-ramp merging problem of automated vehicles (AVs) is mainly designed for a single AV, treating other vehicles as part of the environment. This paper proposes a control framework for cooperative on-ramp merging of multiple AVs based on multi-agent deep reinforcement learning (MADRL). This framework facilitates AVs on the ramp and adjacent mainline to learn a coordinate control policy for their longitudinal and lateral motions based on the environment observations. Unlike the hierarchical architecture, this paper integrates decision and control into a unified optimal control problem to solve an on-ramp merging strategy through MADRL. Firstly, a partially observable Markov game (POMG) is formulated to characterize the on-ramp merging control problem, where the observation space of each AV (agent) is defined as its states and the relative state between it and other AVs, and the joint action spaces are the longitudinal acceleration and front wheel steering angle of AVs. Then, with safety and traffic efficiency as the objective, the reward function of each AV is designed. Furthermore, the joint action for multi-agent is obtained by solving the POMG problem utilizing the multi-agent deep deterministic policy gradient (MADDPG) method. Finally, a rule-based action guidance strategy is presented to supervise further the joint action for enhancing the safety of AVs. Numerical experiments are performed under different conditions to verify the effectiveness of the proposed merging control framework for a multi-agent system. The proposed scheme is also compared with the method for a single agent, taking the deep deterministic policy gradient (DDPG) method as a benchmark. The results demonstrate superior performance of the proposed method than the DDPG method in terms of safety and traffic efficiency.</div></div>
Reference26 articles.
1. He , X. and Lv , C. Toward Intelligent Connected E-Mobility: Energy-Aware Cooperative Driving with Deep Multiagent Reinforcement Learning IEEE Vehicular Technology Magazine 10.1109/MVT.2023.3291171
2. Jin , W. , Islam , M. , and Chowdhury , M. Risk-Based Merging Decisions for Autonomous Vehicles Journal of Safety Research 83 2022 45 56
3. Hu , J. , Li , X. , Cen , Y. , Xu , Q. et al. A Roadside Decision-Making Methodology Based on Deep Reinforcement Learning to Simultaneously Improve the Safety and Efficiency of Merging Zone IEEE Transactions on Intelligent Transportation Systems 23 10 2022 18620 18631
4. He , X. , Lou , B. , Yang , H. , and Lv , C. Robust Decision Making for Autonomous Vehicles at Highway on-Ramps: A Constrained Adversarial Reinforcement Learning Approach IEEE Transactions on Intelligent Transportation Systems 24 4 2022 4103 4113
5. Li , G. , Yang , Y. , Li , S. , Qu , X. et al. Decision Making of Autonomous Vehicles in Lane Change Scenarios: Deep Reinforcement Learning Approaches with Risk Awareness Transportation Research Part C: Emerging Technologies 134 2022 103452