Energy-Efficient Driving for Adaptive Traffic Signal Control Environment via Explainable Reinforcement Learning

Abstract

Energy-efficient driving systems can effectively reduce energy consumption during vehicle operation. Most of the existing studies focus on the driving strategies in a fixed signal timing environment, whereas the standardized Signal Phase and Timing (SPaT) data can help the vehicle make the optimal decisions. However, with the development of artificial intelligence and communication techniques, the conventional fixed timing methods are gradually replaced by adaptive traffic signal control (ATSC) approaches. The previous studies utilized SPaT information that cannot be applied directly in the environment with ATSC. Thus, a framework is proposed to implement energy-efficient driving in the ATSC environment, while the ATSC is realized by the value-based reinforcement learning algorithm. After giving the optimal control model, the framework draws upon the Markov Decision Process (MDP) to make an approximation to the optimal control problem. The state sharing mechanism allows the vehicle to obtain the state information of the traffic signal agents. The reward function in MDP considers energy consumption, traffic mobility, and driving comfort. With the support of traffic simulation software SUMO, the vehicle agent is trained by Proximal Policy Optimization (PPO) algorithm, which enables the vehicle to select actions from continuous action space. The simulation results show that the energy consumption of the controlled vehicle can be reduced by 31.73%~45.90% with a different extent of mobility sacrifice compared with the manual driving model. Besides, we developed a module based on SHapley Additive exPlanations (SHAP) to explain the decision process in each timestep of the vehicle. That can make the strategy more reliable and credible.