A Deep Q-Learning Network for Dynamic Constraint-Satisfied Service Composition

Abstract

Traditional service composition methods usually address the constraint-satisfied service composition (CSSC) problem with static web services. Such solutions however are inapplicable to the dynamic scenarios where the services or their QoS values may change over time. Some recent studies are proposed to use reinforcement learning, especially, integrate the idea of Q-learning, to solve the dynamic CSSC problem. However, such Q-learning algorithm relies on Q-table to search for optimal candidate services. When the problem of CSSC becomes complex, the number of states in Q-table is very large and the cost of the Q-learning model will become extremely high. In this paper, the authors propose a novel solution to address this issue. By training a DQN network to replace the Q-table, this solution can effectively model the uncertainty of services with fine-grained QoS attributes and choose suitable candidate services to compose on the fly in the dynamic scenarios. Experimental results on both artificial and real datasets demonstrate the effectiveness of the method.