Improvement of the Control of a Grid Connected Photovoltaic System Based on Synergetic and Sliding Mode Controllers Using a Reinforcement Learning Deep Deterministic Policy Gradient Agent

Abstract

This article presents the control of a grid connected PV (GC-PV) array system, starting from a benchmark. The control structure used in this article was a cascade-type structure, in which PI or synergetic (SYN) controllers were used for the inner control loop of id and iq currents and PI or sliding mode control (SMC) controllers were used for the outer control loop of the udc voltage from the DC intermediate circuit. This paper presents the mathematical model of the PV array together with the main component blocks: simulated inputs for the PV array; the PV array itself; the MPPT algorithm; the DC-DC boost converter; the voltage and current measurements for the DC intermediate circuit; the load and connection to power grid; the DC-AC converter; and the power grid. It also presents the stages of building and training the reinforcement learning (RL) agent. To improve the performance of the control system for the GC-PV array system without using controllers with a more complicated mathematical description, the advantages provided by the RL agent on process controls could also be used. This technique does not require exact knowledge of the mathematical model of the controlled system or the type of uncertainties. The improvement in the control system performance for the GC-PV array system, both when using simple PI-type controllers or complex SMC- and SYN-type controllers, was achieved using an RL agent based on the Deep Deterministic Policy Gradient (DDPG). The variant of DDPG used in this study was the Twin-Delayed (TD3). The improvement in performance of the control system were obtained by using the correction command signals provided by the trained RL agent, which were added to the command signals ud, uq and idref. The parametric robustness of the proposed control system based on SMC and SYN controllers for the GC-PV array system was proven in the case of a variation of 30% caused by the three-phase load. Moreover, the results of the numerical simulations are shown comparatively and the validation of the synthesis of the proposed control system was obtained. This was achieved by comparing the proposed system with a software benchmark for the control of a GC-PV array system performed in MATLAB Simulink. The numerical simulations proved the superiority of the performance of control systems that use the RL-TD3 agent.