Parametric Dueling DQN- and DDPG-Based Approach for Optimal Operation of Microgrids

Abstract

This study is aimed at addressing the problem of optimizing microgrid operations to improve local renewable energy consumption and ensure the stability of multi-energy systems. Microgrids are localized power systems that integrate distributed energy sources, storage, and controllable loads to enhance energy efficiency and reliability. The proposed approach introduces a novel microgrid optimization method that leverages the parameterized Dueling Deep Q-Network (Dueling DQN) and Deep Deterministic Policy Gradient (DDPG) algorithms. The method employs a parametric hybrid action-space reinforcement learning technique, where the DDPG is utilized to convert discrete actions into continuous action values corresponding to each discrete action, while the Dueling DQN uses the current observation states and these continuous action values to predict the discrete actions that maximize Q-values. This integrated strategy is designed to tackle the co-scheduling challenge in microgrids, enabling them to dynamically select the most favorable control strategies based on their specific states and the actions of other intelligent entities. The ultimate objective is to minimize the overall operational costs of microgrids while ensuring the efficient local consumption of renewable energy and maintaining the stability of multi-energy systems. Simulation experiments were conducted to validate the efficacy and superiority of the proposed method in achieving the optimal microgrid operation, showcasing its potential to improve service quality and reduce operational expenses. Average rewards increased by 30% and 15% compared to the use of the Dueling DQN or DDPG only.