Dynamic Capacitated Arc Routing Problem in E-Bike Sharing System: A Monte Carlo Tree Search Approach

Abstract

This paper studies a dynamic capacitated arc routing problem for battery replacement in an e-bike sharing system, where workers replace batteries for underpowered e-bikes along street segments dynamically. The objective is to replace as many batteries as possible and minimize pickup failures. The temporal dependency of the routing decisions, the conflict of the workers’ operations, and the stochastic and dynamic nature of user demands all make this a difficult problem. To cope with these difficulties, a “Partition-First, Route-Second” bi-level solution framework is adopted to describe the problem in two different time scales. On the large time scale, a spatiotemporal partitioning method, which divides the road network into nonoverlapping subzones, is proposed to decompose the problem. On the small time scale, this paper models the routing decision process of individual worker as a Markov Decision Process. We adopt a lookahead policy that simulates future information and decisions over some horizons to evaluate the long-term influence of current feasible decisions. A Monte Carlo Tree Search algorithm is also used to improve the simulation efficiency. By performing numerical computation experiments on a test case study and comparing with some benchmarking policies, we demonstrate the effectiveness and efficiency of the suggested method.