Comparison of Reinforcement Learning Agents Applied to Traffic Signal Optimisation

Abstract

Traditional methods for traffic signal control at an urban intersection are not effective in controlling traffic flow for dynamic traffic demand which leads to negative environmental, psychological and financial impacts for all parties involved. Urban traffic management is a complex problem with multiple factors  effecting the control of traffic flow. With recent advancements in machine learning (ML), especially reinforcement learning (RL), there is potential to solve this problem. The idea is to allow an agent to learn optimal behaviour to maximise specific metrics through trial and error. In this paper we apply two RL algorithms, one policy-based, the other value-based, to solve this problem in simulation. For the simulation, we use an open-source traffic simulator, Simulation of Urban MObility (SUMO), packaged as an OpenAI Gym environment. We trained the agents on different traffic patterns on a simulated intersection. We compare the performance of the resultant policies to traditional approaches such as the Webster and vehicle actuated (VA) methods. We also examine and  contrast the policies learned by the RL agents and evaluate how well they generalise to different traffic patterns.