Robust Multiagent Reinforcement Learning for UAV Systems: Countering Byzantine Attacks
-
Published:2023-11-19
Issue:11
Volume:14
Page:623
-
ISSN:2078-2489
-
Container-title:Information
-
language:en
-
Short-container-title:Information
Author:
Medhi Jishu K.1ORCID, Liu Rui1, Wang Qianlong2ORCID, Chen Xuhui1ORCID
Affiliation:
1. College of Aeronautics and Engineering, Kent State University, Kent, OH 44240, USA 2. Department of Computer & Information Sciences, Towson University, Towson, MD 21252, USA
Abstract
Multiple unmanned aerial vehicle (multi-UAV) systems have gained significant attention in applications, such as aerial surveillance and search and rescue missions. With the recent development of state-of-the-art multiagent reinforcement learning (MARL) algorithms, it is possible to train multi-UAV systems in collaborative and competitive environments. However, the inherent vulnerabilities of multiagent systems pose significant privacy and security risks when deploying general and conventional MARL algorithms. The presence of even a single Byzantine adversary within the system can severely degrade the learning performance of UAV agents. This work proposes a Byzantine-resilient MARL algorithm that leverages a combination of geometric median consensus and a robust state update model to mitigate, or even eliminate, the influence of Byzantine attacks. To validate its effectiveness and feasibility, the authors include a multi-UAV threat model, provide a guarantee of robustness, and investigate key attack parameters for multiple UAV navigation scenarios. Results from the experiments show that the average rewards during a Byzantine attack increased by up to 60% for the cooperative navigation scenario compared with conventional MARL techniques. The learning rewards generated by the baseline algorithms could not converge during training under these attacks, while the proposed method effectively converged to an optimal solution, proving its viability and correctness.
Subject
Information Systems
Reference44 articles.
1. Wise, R., and Rysdyk, R. (2008, January 18–21). UAV coordination for autonomous target tracking. Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, Honolulu, HI, USA. 2. Elloumi, M., Dhaou, R., Escrig, B., Idoudi, H., and Saidane, L.A. (2018, January 15–18). Monitoring road traffic with a UAV-based system. Proceedings of the 2018 IEEE Wireless Communications and Networking Conference (WCNC), Barcelona, Spain. 3. Merino, L., Caballero, F., Martinez-de Dios, J., and Ollero, A. (2005, January 18–22). Cooperative fire detection using unmanned aerial vehicles. Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain. 4. Crop monitoring using unmanned aerial vehicles: A review;Cuaran;Agric. Rev.,2021 5. Scherer, J., Yahyanejad, S., Hayat, S., Yanmaz, E., Andre, T., Khan, A., Vukadinovic, V., Bettstetter, C., Hellwagner, H., and Rinner, B. (2015, January 18). An autonomous multi-UAV system for search and rescue. Proceedings of the First Workshop on Micro Aerial Vehicle Networks, Systems, and Applications for Civilian Use, Florence, Italy.
|
|