TD3-Based EMS Using Action Mask and Considering Battery Aging for Hybrid Electric Dump Trucks

Abstract

The hybrid electric dump truck is equipped with multiple power sources, and each powertrain component is controlled by an energy management strategy (EMS) to split the demanded power. This study proposes an EMS based on deep reinforcement learning (DRL) algorithm to extend the battery life and reduced total usage cost for the vehicle, namely the twin delayed deep deterministic policy gradient (TD3) based EMS. Firstly, the vehicle model is constructed and the optimization objective function, including battery aging cost and fuel consumption cost, is designed. Secondly, the TD3-based EMS is used for continuous action control of ICE power based on vehicle state, and the action mask is applied to filter out invalid actions. Thirdly, the simulations of the EMSs are trained under the CHTC-D driving cycle and C-WTVC driving cycle. The results show that the action mask improves the convergence efficiency of the strategies, and the proposed TD3-based EMS outperforms the deep deterministic policy gradient (DDPG) based EMS. Meanwhile, the battery life is extended by 36.17% under CHTC-D and 35.49% under C-WTVC, and the total usage cost is reduced by 4.30% and 2.49% when the EMS considers battery aging. In summary, the proposed TD3-based EMS can extend the battery life and reduce usage cost, and provides a method to solve the optimization problem for the EMS of hybrid power systems.