The synergy of the multi-modal MPC and Q-learning approach for the navigation of a three-wheeled omnidirectional robot based on the dynamic model with obstacle collision avoidance purposes

Abstract

This paper proposes a new navigation control scheme for a three-wheel OMnidirectional Robot (OMR), taking into account the dynamic constraints with the aim of collision avoidance in dynamic environments. The proposed control strategy is inherently a multi-modal adaptive-Reinforcing model-based controller (MAR-MPC), which can be regarded as a judicious synergy of multi-modal MPC (MMPC) and Q-Learning (QL) to optimal navigation. This method takes advantage of a larger convergence area, compared to MMPC and MPC, and presents near-optimal MPC performance. The navigation scheme utilizes an online stochastic observer, namely, a Kalman filter, to estimate the future state of the robot. This paper formulates a general collision avoidance navigation problem in a constrained linear convex cone structure to make it real-time implementable. Furthermore, the proposed multi-modal path planning algorithm reduces the required prediction horizon and consequently affects computational cost. To evaluate the performance of the proposed navigation strategy, two static and two dynamic environment instances are simulated, and the results are compared with four algorithms, namely, Exploring Random Tree (RRT), Potential Field (PF), MPC, and MMPC. Results indicate the superior performance of the proposed navigation method in aspects of time and distance costs and collision avoidance criterion. Moreover, the proposed algorithm provides the feasibility and stability guarantee and a larger convergence region.