Revealing principles of autonomous thermal soaring in windy conditions using vulture-inspired deep reinforcement-learning

Abstract

Thermal soaring, a technique used by birds and gliders to utilize updrafts of hot air, presents an attractive model for developing biomimetic autonomous and unmanned aerial vehicles (UAVs) capable of long-endurance flight. Previous studies have employed machine- and deep-learning models to control gliding UAVs in simplified environments without horizontal winds. The resulting neural network models operate as ‘black boxes’, with little insight into their navigation policy or the structure of the problem. Here, we present a deep reinforcement-learning framework for autonomous glider control in a simulated environment with thermal updrafts and challenging horizontal winds. Compared with vulture flight data, the resulting autonomous agent spontaneously adopted a vulture-like soaring technique that successfully and robustly exploited thermal updrafts under horizontal winds up to 5 m/sec. This system enabled us to reveal the underlying structure of the thermal soaring problem, which consists of two critical bottlenecks that should be solved sequentially: achieving stable flight and flying near the thermal center. Additionally, the agent’s neural network divides into functional clusters that correlate with distinct behavioral modes during thermal searching and soaring. Our findings may contribute to the development of biomimetic UAVs with vulture-like efficiency and to understanding the structure and bottlenecks of other motion-based problems.