Request delay and survivability optimization for software defined‐wide area networking (SD‐WAN) using multi‐agent deep reinforcement learning

Abstract

AbstractData exchange between headquarters and local branches represents a major challenge issue for business success. For this issue, traditional solutions applied to wide area networks (WAN) remain unrealistic and require a good knowledge of the systems. Recently, software‐defined wide area networking (SD‐WAN) plays a pivotal role and constitutes, in general, a reliable solution for wide area networking. Compared to the classical WAN, SD‐WAN decouples the control plane from gateway devices. Moreover, SD‐WAN are based on centralized WAN management with dynamic reconfiguration, allowing it to control the entire service requirements. Nevertheless, traffic management is the most usual metric in SD‐WAN. It gives a clear indication on propagation latency, request delay and survivability; that is, network connectivity. In this article, to guarantee an efficient quality of service and deal with the increased delay problem, we investigate the problem of joint optimization of average request delay and survivability in the proposed SD‐WAN model. The optimization approaches are formulated so as to minimize the average request delay and maximize the network connectivity. Subsequently, we adopt a multi‐agent deep Q‐Network algorithm to solve it, where the reward function is reformulated based on the optimization objective. Simulation results show that our strategy improves the system's performance significantly in terms of request delay and survivability, compared to traditional baseline algorithms.