Digital twin and deep reinforcement learning enabled real-time scheduling for complex product flexible shop-floor

Abstract

Real-time scheduling methods are essential and critical to complex product flexible shop-floor due to the dynamic events in the production process, such as new job insertions, machine breakdowns and frequent rework. Recently, digital twin (DT) technology can help identify disturbances by continuously comparing physical space with virtual space, which enables real-time scheduling and greatly reduces the deviation between pre-schedule and actual schedule. However, the conventional scheduling models and algorithms cannot satisfy the adaptiveness and timeliness requirements of optimization in DT enabled shop-floor (DTS). To address above challenges, an overall framework of DT enabled real-time scheduling (DTE-RS) for complex product shop-floor is proposed to effectively reduce adverse impacts of the dynamic disturbances and minimize the makespan. Firstly, complex product flexible job shop scheduling problem (CPFJSP) is formulated as Markov Decision Process (MDP), taking into account machine breakdown and new job insertions. Then, deep Q-network (DQN) based solution is developed to achieve optimal task dispatching according to real-time production state. Finally, the case study for aircraft overhaul shop-floor is conducted to demonstrate effectiveness and feasibility of the proposed real-time scheduling method. Through experimental comparison, it is indicated that the proposed method could effectively respond to dynamic disturbances and outperform the dynamic scheduling method in terms of makespan.