Sarsa(Λ)-Based Logistics Planning Approximated by Value Function with Policy Iteration

Abstract

The logistics planning problem has been extensively investigated for a long time. However, with the increasing number of stochastic events occurred in road, increasing number of stochastic factors should be taken into consideration. A dynamic approach is used in this paper to solve the logistics planning problem in the common form of stochastic demand with the reinforcement learning framework which is able to optimize policy in unknown environments and uncertain cases. We take advantage of clustering method to extract states as main features for basis function so as to solve the dimensionality curse problems caused by stochastic settings. We also propose an approximation approach with the policy iteration restricted by the goal of minimal time differential error to approximate the stochastic cases of the real world, and then use the attained approximation parameters as input for the proposed Sarsa(Λ)-based logistics planning algorithm to determine the policy and action in accordance with the real world stochastic events. The benchmarking experimental results showed that the proposed algorithm has achieved improvements in almost all the test cases.