Optimal Scheduling of Energy Storage System Considering Life-Cycle Degradation Cost Using Reinforcement Learning

Abstract

Recently, due to the ever-increasing global warming effect, the proportion of renewable energy sources in the electric power industry has increased significantly. With the increase in distributed power sources with adjustable outputs, such as energy storage systems (ESSs), it is necessary to define ESS usage standards for an adaptive power transaction plan. However, the life-cycle cost is generally defined in a quadratic formula without considering various factors. In this study, the life-cycle cost for an ESS is defined in detail based on a life assessment model and used for scheduling. The life-cycle cost is affected by four factors: temperature, average state-of-charge (SOC), depth-of-discharge (DOD), and time. In the case of the DOD stress model, the life-cycle cost is expressed as a function of the cycle depth, whose exact value can be determined based on fatigue analysis techniques such as the Rainflow counting algorithm. The optimal scheduling of the ESS is constructed considering the life-cycle cost using a tool based on reinforcement learning. Since the life assessment cannot apply the analytical technique due to the temperature characteristics and time-dependent characteristics of the ESS SOC, the reinforcement learning that derives optimal scheduling is used. The results show that the SOC curve changes with respect to weight. As the weight of life-cycle cost increases, the ESS output and charge/discharge frequency decrease.