Pre- and post-processing techniques for reinforcement-learning-based routing and spectrum assignment in elastic optical networks

Abstract

Research on the routing and spectrum assignment (RSA) problem has long been conducted with the aim of efficiently utilizing the frequency resources of optical networks. Given the recent progress in machine learning (ML) technology, it has been reported that the application of ML in various areas of optical network design, operation, and management has the potential to bring about new innovations such as autonomous optical network operation and highly accurate estimation of network conditions. With regard to the RSA problem, it is expected that an algorithm that achieves better accommodation efficiency than conventional heuristic methods can be realized by applying reinforcement learning (RL), which well supports training in a simulation environment. In this paper, we introduce and evaluate three techniques devised to apply RL more effectively to elastic optical networks (EONs): two pre-processing techniques called link-axis positional encoding (LPE) and slot-axis positional encoding (SPE) and a post-processing technique named the assignable boundary slot mask. First, we build a simple model in which the state data of the optical network, including frequency slot utilization, are input to the neural network of the RSA agent in RL and show that this model has difficulty outperforming the conventional heuristic RSA algorithm. Next, we build an RSA agent model with the proposed techniques and simulate the accommodation of dynamic optical paths to quantitatively demonstrate that the blocking probability can be reduced by 17% compared to the conventional heuristic.