Multi-Objective Optimization in Air-to-Air Communication System Based on Multi-Agent Deep Reinforcement Learning
Author:
Lin Shaofu1ORCID, Chen Yingying1, Li Shuopeng1ORCID
Affiliation:
1. Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
Abstract
With the advantages of real-time data processing and flexible deployment, unmanned aerial vehicle (UAV)-assisted mobile edge computing systems are widely used in both civil and military fields. However, due to limited energy, it is usually difficult for UAVs to stay in the air for long periods and to perform computational tasks. In this paper, we propose a full-duplex air-to-air communication system (A2ACS) model combining mobile edge computing and wireless power transfer technologies, aiming to effectively reduce the computational latency and energy consumption of UAVs, while ensuring that the UAVs do not interrupt the mission or leave the work area due to insufficient energy. In this system, UAVs collect energy from external air-edge energy servers (AEESs) to power onboard batteries and offload computational tasks to AEESs to reduce latency. To optimize the system’s performance and balance the four objectives, including the system throughput, the number of low-power alarms of UAVs, the total energy received by UAVs and the energy consumption of AEESs, we develop a multi-objective optimization framework. Considering that AEESs require rapid decision-making in a dynamic environment, an algorithm based on multi-agent deep deterministic policy gradient (MADDPG) is proposed, to optimize the AEESs’ service location and to control the power of energy transfer. While training, the agents learn the optimal policy given the optimization weight conditions. Furthermore, we adopt the K-means algorithm to determine the association between AEESs and UAVs to ensure fairness. Simulated experiment results show that the proposed MODDPG (multi-objective DDPG) algorithm has better performance than the baseline algorithms, such as the genetic algorithm and other deep reinforcement learning algorithms.
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference38 articles.
1. Kouadio, L., El Jarroudi, M., Belabess, Z., Laasli, S.-E., Roni, M.Z.K., Amine, I.D.I., Mokhtari, N., Mokrini, F., Junk, J., and Lahlali, R. (2023). A Review on UAV-Based Applications for Plant Disease Detection and Monitoring. Remote Sens., 15. 2. Karlinsky, L., Michaeli, T., and Nishino, K. (2023). Computer Vision–ECCV 2022 Workshops, Springer Nature Switzerland. 3. Fu, W., Gu, M., and Niu, Y. (2023). Proceedings of the 2022 International Conference on Autonomous Unmanned Systems (ICAUS 2022), Xi’an, China, 23–25 September 2022, Springer Nature. 4. Fu, W., Gu, M., and Niu, Y. (2023). Proceedings of the 2022 International Conference on Autonomous Unmanned Systems (ICAUS 2022), Xi’an, China, 23–25 September 2022, Springer Nature. 5. Device-Enhanced MEC: Multi-Access Edge Computing (MEC) Aided by End Device Computation and Caching: A Survey;Mehrabi;IEEE Access,2019
|
|