Advantage Actor-Critic for Autonomous Intersection Management

Abstract

With increasing urban population, there are more and more vehicles, causing traffic congestion. In order to solve this problem, the development of an efficient and fair intersection management system is an important issue. With the development of intelligent transportation systems, the computing efficiency of vehicles and vehicle-to-vehicle communications are becoming more advanced, which can be used to good advantage in developing smarter systems. As such, Autonomous Intersection Management (AIM) proposals have been widely discussed. This research proposes an intersection management system based on Advantage Actor-Critic (A2C) which is a type of reinforcement learning. This method can lead to a fair and efficient intersection resource allocation strategy being learned. In our proposed approach, we design a reward function and then use this reward function to encourage a fair allocation of intersection resources. The proposed approach uses a brake-safe control to ensure that autonomous moving vehicles travel safely. An experiment is performed using the SUMO simulator to simulate traffic at an isolated intersection, and the experimental performance is compared with Fast First Service (FFS) and GAMEOPT in terms of throughput, fairness, and maximum waiting time. The proposed approach increases fairness by 20% to 40%, and the maximum waiting time is reduced by 20% to 36% in high traffic flow. The inflow rates are increased, average waiting time is reduced, and throughput is increased.