Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

Abstract

To adapt to the development trend of intelligent air combat, it is necessary to research the autonomous generation of maneuvering decisions for unmanned combat aerial vehicles (UCAV). This paper presents a maneuver decision-making method for UCAV based on a hybridization of deep Q-network (DQN) and extended Kalman filtering (EKF). Firstly, a three-dimensional air combat simulation environment is constructed, and a flight motion model of UCAV is designed to meet the requirements of the simulation environment. Secondly, we evaluate the current situation of UCAV based on their state variables in air combat, for further network learning and training to obtain the optimal maneuver strategy. Finally, based on the DQN, the system state equation is constructed using the uncertain parameter values of the current network, and the observation equation of the system is constructed using the parameters of the target network. The optimal parameter estimation value of the DQN is obtained by iteratively updating the solution through EKF. Simulation experiments have shown that this autonomous maneuver decision-making method hybridizing DQN with EKF is effective and reliable, as it can eliminate the opponent and preserve its side.