Generating adversarial deep reinforcement learning -based frequency control of Island City microgrid considering generalization of scenarios

Abstract

The increasing incorporation of new energy sources into power grids introduces significant variability, complicating traditional load frequency control (LFC) methods. This variability can cause frequent load disturbances and severe frequency fluctuations in island city microgrids, leading to increased generation costs. To tackle these challenges, this paper introduces a novel Data knowledge-driven load frequency control (DKD-LFC) method, aimed at optimizing the balance between generation cost and frequency stability in isolated microgrids with high renewable energy integration. The DKD-LFC replaces conventional controllers with agent-based systems, utilizing reinforcement learning for adaptive frequency control in complex environments. A new policy generation algorithm, based on generative adversarial-proximal policy optimization (DAC-PPO), is proposed. This algorithm extends the traditional Actor-Critic framework of the Proximal Policy Optimization (PPO) by incorporating a Discriminator network. This network evaluates whether the input state-action pairs align with current or expert policies, guiding policy updates toward expert policies during training. Such an approach enhances the algorithm’s generalization capability, crucial for effective LFC application in diverse operational contexts. The efficacy of the DKD-LFC method is validated using the isolated island city microgrid LFC model of the China Southern Grid (CSG), demonstrating its potential in managing the complexities of modern power grids.