Motion Planning and Control with Randomized Payloads on Real Robot Using Deep Reinforcement Learning

Abstract

In this study, a unified motion planner with low level controller for continuous control of a differential drive mobile robot under variable payload values is presented. The deep reinforcement agent takes 11-dimensional state vector as input and calculates each wheel’s torque value as a 2-dimensional output vector. These torque values are fed into the dynamic model of the robot, and lastly steering commands are gathered. In previous studies, intersection navigation solutions that uses deep-RL methods, have not been considered with variable payloads. This study is focused specifically on service robotic applications where payload is subject to change. In this study, deep-RL-based motion planning is performed by considering both kinematic and dynamic constraints. According to the simulations in a dynamic environment, the agent successfully navigates to target with 98.2% success rate in test time with unseen payload masses during training. Another agent is also trained without payload randomization for comparison. Results show that our agent outperforms the other agent, that is not aware of its own payload, with more than 40% gap. Our agent is also compared with the Time-to-Collision (TTC) algorithm. It is observed that our agent uses far less time than TTC to accomplish the mission while success rates of two methods are same. Lastly, the proposed method is applied on a real robot in order to show the real-time applicability of the approach.