Joint Situational Assessment‐Hierarchical Decision‐Making Framework for Maneuver Intent Decisions

Abstract

Decision‐making in unmanned combat aerial vehicles (UCAVs) presents a multifaceted challenge because of the complexity and dynamics of the flight environment, which leads to hurdles in training convergence, low decision validity, and the dimensionality catastrophe for decision‐making neural networks. A novel framework is proposed to address breaking down the complicated decision issues, which combines the strengths of graph convolutional networks in relation extraction with the ability of hierarchical reinforcement learning. To solve the problem of decision validity under high‐dimensional inputs, the joint framework is applied to the Maneuver Intent's decision, and a maneuver library‐based state space design method is suggested. The joint framework executes adaptable strategies and flight maneuvers to address the issue of training non‐convergence or task failure due to difficult‐to‐obtain reward signals across various scenarios. Then, the recurrent curriculum training and cross‐entropy rewards are designed to train decisions on different sub‐strategies. The experimental evaluation demonstrated more flexibility and adaptability in decision‐making problems under complex tasks compared to rule‐based and reinforcement learning baseline methods. The method proposed in this article provides a novel approach to resolving intricate decision problems, and which has certain theoretical significance and reference value for engineering applications.