Multi objective factory layout planning using simulation-based reinforcement learning

Abstract

Abstract Factory layout planning aims at finding an optimized layout configuration under consideration of varying influences such as the material flow characteristics. Manual layout planning can be characterized as a complex decision-making process due to a large number of possible placement options. Automated planning approaches aim at reducing the manual planning effort by generating optimized layout variants in the early stages of layout planning. Recent developments have introduced Reinforcement Learning (RL) based planning approaches that allow to optimize a layout under consideration of a single optimization criterion. However, within layout planning, multiple partially conflicting planning objectives have to be considered. Such multiple objectives are not considered by existing RL-based approaches. This paper addresses this research gap by presenting a novel RL-based layout planning approach that allows consideration of multiple objectives for optimization. Furthermore, existing RL-based planning approaches only consider analytically formulated objectives such as the transportation distance. Consequently, dynamic influences in the material flow are neglected which can result in higher operational costs of the future factory. To address this issue, a discrete event simulation module is developed that allows simulating manufacturing and material flow processes simultaneously for any layout configuration generated by the RL approach. Consequently, the presented approach considers material flow simulation results for multi-objective optimization. In order to investigate the capabilities of RL-based factory layout planning, different RL architectures are compared based on a simplified application scenario. In terms of optimization objectives, the throughput time, media supply, and clarity of the material flow are considered. The best performing architecture is then applied to an industrial planning scenario with 43 functional units to illustrate the approach. Furthermore, the performance of the RL approach is compared to the manually planned layout and to the results generated by a combined version of the genetic algorithm and tabu search. The results indicate that the RL approach is capable of improving the manually planned layout significantly. Furthermore, it reaches comparable results for the throughput time and better results for the clarity of the material flow compared to the combined version of a genetic algorithm and tabu search.