Abstract
To solve the maneuvering decision problem in air combat of unmanned combat aircraft vehicles (UCAVs), in this paper, an autonomous maneuver decision method is proposed for a UCAV based on deep reinforcement learning. Firstly, the UCAV flight maneuver model and maneuver library of both opposing sides are established. Then, considering the different state transition effects of various actions when the pitch angles of the UCAVs are different, the 10 state variables including the pitch angle, are taken as the state space. Combined with the air combat situation threat assessment index model, a two-layer reward mechanism combining internal reward and sparse reward is designed as the evaluation basis of reinforcement learning. Then, the neural network model of the full connection layer is built according to an Asynchronous Advantage Actor–Critic (A3C) algorithm. In the way of multi-threading, our UCAV keeps interactively learning with the environment to train the model and gradually learns the optimal air combat maneuver countermeasure strategy, and guides our UCAV to conduct action selection. The algorithm reduces the correlation between samples through multi-threading asynchronous learning. Finally, the effectiveness and feasibility of the method are verified in three different air combat scenarios.
Funder
National Natural Science Foundation of China
Basic Research Programs of Taicang, 2021 under Grant
Fun damental Research Funds for the Central Universities
Industrial Development and Foster Project of Yangtze River Delta Research Institute of NPU, Taicang
Subject
Electrical and Electronic Engineering,Industrial and Manufacturing Engineering,Control and Optimization,Mechanical Engineering,Computer Science (miscellaneous),Control and Systems Engineering
Reference26 articles.
1. Azar, A.T., Koubaa, A., and Mohamed, N.A. Drone Deep Reinforcement Learning: A Review. Electronics, 2021. 10.
2. Editorial of Special Issue on UAV Autonomous, Intelligent and Safe Control;Zhang;Guid. Navig. Control.,2022
3. Burgin, G.H. Improvements to the Adaptive Maneuvering Logic Program, 1986.
4. UAV Air Combat Decision Based on Evolutionary Expert System Tree;Wang;Ordnance Ind. Autom.,2019
5. An UAV Air-combat Decision Expert System based on Receding Horizon Contro;Fu;J. Beijing Univ. Aeronaut. Astronaut.,2015
Cited by
13 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献