A Meta Reinforcement Learning Approach for SFC Placement in Dynamic IoT-MEC Networks

Abstract

In order to achieve reliability, security, and scalability, the request flow in the Internet of Things (IoT) needs to pass through the service function chain (SFC), which is composed of series-ordered virtual network functions (VNFs), then reach the destination application in multiaccess edge computing (MEC) for processing. Since there are usually multiple identical VNF instances in the network and the network environment of IoT changes dynamically, placing the SFC for the IoT request flow is a significant challenge. This paper decomposes the dynamic SFC placement problem of the IoT-MEC network into two subproblems: VNF placement and path determination of routing. We first formulate these two subproblems as Markov decision processes. We then propose a meta reinforcement learning and fuzzy logic-based dynamic SFC placement approach (MRLF-SFCP). The MRLF-SFCP contains an inner model that focuses on making SFC placement decisions and an outer model that focuses on learning the initial parameters considering the dynamic IoT-MEC environment. Specifically, the approach uses fuzzy logic to pre-evaluate the link status information of the network by jointly considering available bandwidth, delay, and packet loss rate, which is helpful for model training and convergence. In comparison to existing algorithms, simulation results demonstrate that the MRLF-SFCP algorithm exhibits superior performance in terms of traffic acceptance rate, throughput, and the average reward.