Aerodynamic optimization of airfoil based on deep reinforcement learning

Abstract

The traditional optimization of airfoils relies on, and is limited by, the knowledge and experience of the designer. As a method of intelligent decision-making, reinforcement learning can be used for such optimization through self-directed learning. In this paper, we use the lift–drag ratio as the objective of optimization to propose a method for the aerodynamic optimization of airfoils based on a combination of deep learning and reinforcement learning. A deep neural network (DNN) is first constructed as a surrogate model to quickly predict the lift–drag ratio of the airfoil, and a double deep Q-network (double DQN) algorithm is then designed based on deep reinforcement learning to train the optimization policy. During the training phase, the agent uses geometric parameters of the airfoil to represent its state, adopts a stochastic policy to generate optimization experience, and uses a deterministic policy to modify the geometry of the airfoil. The DNN calculates changes in the lift–drag ratio of the airfoil as a reward, and the environment constantly feeds the states, actions, and rewards back to the agent, which dynamically updates the policy to retain positive optimization experience. The results of simulations show that the double DQN can learn the general policy for optimizing the airfoil to improve its lift–drag ratio to 71.46%. The optimization policy can be generalized to a variety of computational conditions. Therefore, the proposed method can rapidly predict the aerodynamic parameters of the airfoil and autonomously learn the optimization policy to render the entire process intelligent.