Deep Reinforcement Learning-Based Multipath Routing for LEO Megaconstellation Networks

Abstract

The expansion of megaconstellation networks (MCNs) represents a promising solution for achieving global Internet coverage. To meet the growing demand for satellite services, multipath routing allows the simultaneous establishment of multiple transmission paths, enabling the transmission of flows in parallel. Nevertheless, the mobility of satellites and time-varying link states presents a challenge for the discovery of optimal paths and traffic scheduling in multipath routing. Given the inflexibility of traditional static deep reinforcement learning (DRL)-based routing algorithms in dealing with time-varying constellation topologies, DRL-based multipath routing (DMR) enabled by a graph neural network (GNN) is proposed as a means of enhancing the transmission performance of MCNs. DMR decouples the stochastic optimization problem of multipath routing under traffic and bandwidth constraints into two subproblems: multipath routing discovery and multipath traffic scheduling. Firstly, the minimum hop count-based multipath route discovery algorithm (MHMRD) is proposed for the computation of multiple available paths between all source and destination nodes. Secondly, the GNN-based multipath traffic scheduling scheme (GMTS) is proposed as a means of dynamically scheduling the traffic on each available path for each data stream, based on the state information of ISLs and traffic demand. Simulation results demonstrate that the proposed scheme can be scaled to constellations with different configurations without the necessity for repeated training and enhance the throughput, completion ratio, and delay by 42.64%, 17.39%, and 3.66% in comparison with the shortest path first algorithm (SPF), respectively.