A Virtual Network Embedding Algorithm Based On Double-Layer Reinforcement Learning

Abstract

Abstract Virtual network embedding (VNE) algorithms dominate the effectiveness of resource sharing in network virtualization. Heuristic embedding algorithms generally make embedding decisions by artificially specified strategies, in which the node importance is measured by simply summing or multiplying several node attributes. However, the contributions of different attributes may be combined through complex functional relationships. The reinforcement learning-based VNE algorithms can optimize node embedding. However, the existing algorithms only consider the local node attributes, and only simple shortest path-based embedding policy is adopted for link embedding, resulting limited embedding effects. To overcome the above defects, we propose a double-layer reinforcement learning-based VNE algorithm (DRL-VNE). In DRL-VNE, both the global and local node attributes are extracted to represent the status of network nodes, then a policy network is constructed to optimize node embedding, and the other policy network is designed to optimize link embedding. The performance of DRL-VNE is evaluated under different network scenarios and is compared with that of heuristic and machine learning-based VNE algorithms. Simulation results show that in hierarchical network scenario, the request acceptance ratio and the resource utilization of DRL-VNE are respectively improved by 14% and by 27% compared with the best performance comparison algorithm.