Affiliation:
1. Department of AI & Informatics, Sangmyung University, Seoul 03016, Republic of Korea
2. Department of Human-Centered Artificial Intelligence, Sangmyung University, Seoul 03016, Republic of Korea
Abstract
In services such as metaverse, which should provide a constant quality of service (QoS) regardless of the user’s physical location, the end-to-end (E2E) latency must be fairly distributed over any flow in the network. To this end, we propose a reinforcement learning (RL)-based scheduler for minimizing the maximum network E2E latency. The RL model used the double deep Q-network (DDQN) with the prioritized experience replay (PER). In order to see the performance change according to the type of RL agent, we implemented a single-agent environment where the controller is an agent and a multi-agent environment where each node is an agent. Since the agents were unable to identify E2E latencies in the multi-agent environment, the state and reward were formulated using the estimated E2E latencies. To precisely evaluate the RL-based scheduler, we set out benchmark algorithms to compare with which a network-arrival-time-based heuristic algorithm (NAT-HA) and a maximum-estimated-delay-based heuristic algorithm (MED-HA). The RL-based scheduler, first-in-first-out (FIFO), round-robin (RR), NAT-HA, and MED-HA were compared through large-scale simulations on four network topologies. The simulation results in fixed-packet generation scenarios showed that our proposal, the RL-based scheduler, achieved the minimization of maximum E2E latency in all the topologies. In other scenarios with random flow generation, the RL-based scheduler and MED-HA showed the lowest maximum E2E latency for all topologies. Depending on the topology, the maximum E2E latency of NAT-HA was equal to or larger than that of the RL-based scheduler. In terms of fairness, the RL-based scheduler showed a higher level of fairness than that of FIFO and RR. NAT-HA had similar or lower fairness than the RL-based scheduler depending on the topology, and MED-HA had the same level of fairness as the RL-based scheduler.
Funder
National Research Foundation of Korea
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference34 articles.
1. Denda, R., Banchs, A., and Effelsberg, W. (2000, January 25–26). The Fairness Challenge in Computer Networks. Proceedings of the Quality of Future Internet Services, First COST 263 International Workshop, QofIS 2000, Berlin, Germany.
2. Fairness in Wireless Networks:Issues, Measures and Challenges;Huaizhou;IEEE Commun. Surv. Tutor.,2014
3. (2022, November 17). Segment Routing Architecture, IETF RFC 8402, July 2018. Available online: https://datatracker.ietf.org/doc/rfc8402/.
4. (2022, August 09). Deterministic Networking Architecture, IETF RFC 8655, October 2019. Available online: https://datatracker.ietf.org/doc/rfc8655/.
5. 6LB: Scalable and Application-Aware Load Balancing with Segment Routing;Desmouceaux;IEEE/ACM Trans. Netw.,2018