Multi‐AGV route planning in automated warehouse system based on shortest‐time Q‐learning algorithm

Abstract

AbstractRoute planning for automated guided vehicles (AGVs) is one of the key factors that affects work efficiency of automated storage and retrieval systems (AS/RSes). Route planning plays an important role in the operation of AGVs. Since the characteristic of AS/RSes is chessboard‐like, the environment is more complex than traditional route planning environments because the number of nodes is large, more than one shortest route exists between two nodes, and the routes with the shortest distance may not be the most energy‐saving routes. Although the traditional route planning algorithms such as the classical Q‐learning algorithm can work well in AGV route planning, it also has some limitations. This paper proposes a novel multi‐AGV route planning approach to solving the AGV route planning problem in the chessboard‐like warehouse, which can improve the route planning efficiency greatly. First, by combining adjacency matrix and reward matrix, we propose a low‐dimensional adjacency‐reward matrix for route planning. This algorithm improves the efficiency of classical Q‐learning algorithms and accelerates dynamic route planning significantly. We further improve the algorithm by considering the travel directions to minimize the number of turns in the route and additionally by considering whether the AGV is loaded or not and plan routes accordingly. Finally, we propose a multi‐AGV online collision‐free route planning algorithm based on these considerations for dynamic route planning for multi‐AGVs operating in a large‐scale warehouse. The proposed algorithms are validated with several case studies.