UAV Path Planning Optimization Strategy: Considerations of Urban Morphology, Microclimate, and Energy Efficiency Using Q-Learning Algorithm

Abstract

The use of unmanned aerial vehicles (UAVS) has been suggested as a potential communications alternative due to their fast implantation, which makes this resource an ideal solution to provide support in scenarios such as natural disasters or intentional attacks that may cause partial or complete disruption of telecommunications services. However, one limitation of this solution is energy autonomy, which affects mission life. With this in mind, our group has developed a new method based on reinforcement learning that aims to reduce the power consumption of UAV missions in disaster scenarios to circumvent the negative effects of wind variations, thus optimizing the timing of the aerial mesh in locations affected by the disruption of fiber-optic-based telecommunications. The method considers the K-means to stagger the position of the resource stations—from which the UAVS launched—within the topology of Stockholm, Sweden. For the UAVS’ locomotion, the Q-learning approach was used to investigate possible actions that the UAVS could take due to urban obstacles randomly distributed in the scenario and due to wind speed. The latter is related to the way the UAVS are arranged during the mission. The numerical results of the simulations have shown that the solution based on reinforcement learning was able to reduce the power consumption by 15.93% compared to the naive solution, which can lead to an increase in the life of UAV missions.