Affiliation:
1. College of Marine Electrical Engineering, Dalian Maritime University, Dalian 116026, China
Abstract
To enhance the anti-submarine and search capabilities of multiple Unmanned Aerial Vehicle (UAV) groups in complex marine environments, this paper proposes a flexible action-evaluation algorithm known as Knowledge-Driven Soft Actor-Critic (KD-SAC), which can effectively interact with real-time environmental information. KD-SAC is a reinforcement learning algorithm that consists of two main components: UAV Group Search Knowledge Base (UGSKB) and path planning strategy. Firstly, based on the UGSKB, we establish a cooperation search framework that comprises three layers of information models: the data layer provides prior information and fundamental search rules to the system, the knowledge layer enriches search rules and database in continuous searching processes, and the decision layer utilizes above two layers of information models to enable autonomous decision-making by UAVs. Secondly, we propose a rule-based deductive inference return visit (RDIRV) strategy to enhance the knowledge base of search. The core concept of this strategy is to enable UAVs to learn from both successful and unsuccessful experiences, thereby enriching the search rules based on optimal decisions as exemplary cases. This approach can significantly enhance the learning performance of KD-SAC. The subsequent step involves designing an event-based UGSKB calling mechanism at the decision-making level, which calls a template based on the target and current motion. Finally, it uses a punishment function, and is then employed to achieve optimal decision-making for UAV actions and states. The feasibility and superiority of our proposed algorithm are demonstrated through experimental comparisons with alternative methods. The final results demonstrate that the proposed method achieves a success rate of 73.63% in multi-UAV flight path planning within complex environments, surpassing the other three algorithms by 17.27%, 29.88%, and 33.51%, respectively. In addition, the KD-SAC algorithm outperforms the other three algorithms in terms of synergy and average search reward.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference27 articles.
1. A Survey of Maritime Unmanned Search System: Theory, Applications and Future Directions;Li;Ocean. Eng.,2023
2. Context-Aware Decision Support for Anti-Submarine Warfare Mission Planning within a Dynamic Environment;Mishra;IEEE Trans. Syst. Man Cybern. Syst.,2020
3. Path Planning Optimization in Unmanned Aerial Vehicles Using Meta-heuristic Algorithms: A Systematic Review;Yahia;Environ. Monit. Assess.,2023
4. Li, F. (2022, January 21–23). Technical Research on Scheme Design and Decision of Unmanned Cluster Cooperative Anti-Submarine. Proceedings of the 2022 IEEE 13th International Conference on Software Engineering and Service Science, Beijing, China.
5. Effectiveness of a Camera as a UAV Mounted Search Sensor for Target Detection: An Experimental Investigation;Velpula;Int. J. Control Autom. Syst.,2021