A City-Wide Crowdsourcing Delivery System with Reinforcement Learning

Abstract

The revolution of online shopping in recent years demands corresponding evolution in delivery services in urban areas. To cater to this trend, delivery by the crowd has become an alternative to the traditional delivery services thanks to the advances in ubiquitous computing. Notably, some studies use public transportation for crowdsourcing delivery, given its low-cost delivery network with millions of passengers as potential couriers. However, multiple practical impact factors are not considered in existing public-transport-based crowdsourcing delivery studies due to a lack of data and limited ubiquitous computing infrastructures in the past. In this work, we design a crowdsourcing delivery system based on public transport, considering the practical factors of time constraints, multi-hop delivery, and profits. To incorporate the impact factors, we build a reinforcement learning model to learn the optimal order dispatching strategies from massive passenger data and package data. The order dispatching problem is formulated as a sequential decision making problem for the packages routing, i.e., select the next station for the package. A delivery time estimation module is designed to accelerate the training process and provide statistical delivery time guarantee. Three months of real-world public transportation data and one month of package delivery data from an on-demand delivery platform in Shenzhen are used in the evaluation. Compared with existing crowdsourcing delivery algorithms and widely used baselines, we achieve a 40% increase in profit rates and a 29% increase in delivery rates. Comparison with other reinforcement learning algorithms shows that we can improve the profit rate and the delivery rate by 9% and 8% by using time estimation in action filtering. We share the data used in the project to the community for other researchers to validate our results and conduct further research.1 [1].