Factory Simulation of Optimization Techniques Based on Deep Reinforcement Learning for Storage Devices

Abstract

In this study, reinforcement learning (RL) was used in factory simulation to optimize storage devices for use in Industry 4.0 and digital twins. Industry 4.0 is increasing productivity and efficiency in manufacturing through automation, data exchange, and the integration of new technologies. Innovative technologies such as the Internet of Things (IoT), artificial intelligence (AI), and big data analytics are smartly automating manufacturing processes and integrating data with production systems to monitor and analyze production data in real time and optimize factory operations. A digital twin is a digital model of a physical product or process in the real world. It is built on data and real-time information collected through sensors and accurately simulates the behavior and performance of a real-world manufacturing floor. With a digital twin, one can leverage data at every stage of product design, development, manufacturing, and maintenance to predict, solve, and optimize problems. First, we defined an RL environment, modeled it, and validated its ability to simulate a real physical system. Subsequently, we introduced a method to calculate reward signals and apply them to the environment to ensure the alignment of the behavior of the RL agent with the task objective. Traditional approaches use simple reward functions to tune the behavior of reinforcement learning agents. These approaches issue rewards according to predefined rules and often use reward signals that are unrelated to the task goal. However, in this study, the reward signal calculation method was modified to consider the task goal and the characteristics of the physical system and calculate more realistic and meaningful rewards. This method reflects the complex interactions and constraints that occur during the optimization process of the storage device and generates more accurate episodes of reinforcement learning in agent behavior. Unlike the traditional simple reward function, this reflects the complexity and realism of the storage optimization task, making the reward more sophisticated and effective.The stocker simulation model was used to validate the effectiveness of RL. The model is a storage device that simulates logistics in a manufacturing production area. The results revealed that RL is a useful tool for automating and optimizing complex logistics systems, increasing the applicability of RL in logistics. We proposed a novel method for creating an agent through learning using the proximal policy optimization algorithm, and the agent was optimized by configuring various learning options. The application of reinforcement learning resulted in an effectiveness of 30–100%, and the methods can be expanded to other fields.