Data-efficient deep reinforcement learning with expert demonstration for active flow control

Abstract

Deep reinforcement learning (RL) is capable of identifying and modifying strategies for active flow control. However, the classic active formulation of deep RL requires lengthy active exploration. This paper describes the introduction of expert demonstration into a classic off-policy RL algorithm, the soft actor-critic algorithm, for application to vortex-induced vibration problems. This combined online-learning framework is applied to an oscillator wake environment and a Navier–Stokes environment with expert demonstration obtained from the pole-placement method and surrogate model optimization. The results show that the soft actor-critic framework combined with expert demonstration enables rapid learning of active flow control strategies through a combination of prior demonstration data and online experience. This study develops a new data-efficient RL approach for discovering active flow control strategies for vortex-induced vibration, providing a more practical methodology for industrial applications.