Research on load frequency control of multi-microgrids in an isolated system based on the multi-agent soft actor-critic algorithm

Abstract

Abstract Due to the limited capacity of a single microgrid, multiple sub-microgrids form interconnected multi-microgrids. However, load variation, distributed power output uncertainty and multi-microgrids network complexity have brought great difficulties to the frequency stability of the whole microgrid. To address this problem, this paper uses a multi-agent deep reinforcement learning（DRL）algorithm to design the controllers to control the frequency of the multi-microgrids. Firstly, a Load Frequency Control (LFC) model for multi-microgrids was built for a single microgrid. Secondly, based on the Centralized Training and Decentralized execution (CTDE) multi-agent reinforcement learning (RL) framework, the Multi-Agent Soft Actor-Critic (MASAC) algorithm was designed and applied to the multi-microgrids model. The state space and action space of multi-agent were established according to the frequency deviation of every sub-microgrid and the output of each distributed power source. The reward function was then established according to the frequency deviation, and the frequency control problem was transformed into the reward maximization problem. The appropriate neural network and training parameters were selected to generate the interconnected microgrid controllers through multiple training of pre-learning. Finally, the simulation study shows that the MASAC controller proposed in this paper can quickly maintain frequency stability when the system is disturbed. The MASAC controller has strong adaptability and robustness under complex operating conditions whence the wind turbine is incapable of frequency regulation and the distribution network of the isolated system changes.