A visual servo reinforcement learning control of uncalibrated manipulators with multi-channel gain decision

Abstract

A technology based on Kalman filtering method combined with multi-channel gain training reinforcement learning for uncalibrated camera visual servo tasks is proposed in this paper. First, a dynamic system with state variables formed from the elements of the image Jacobian matrix is constructed to describe the mapping relationship between two-dimensional images and three-dimensional poses. Kalman filter is used to estimate the state variables of the constructed system online. Next, the Jacobian matrix estimation and depth determination strategy gradient (DDPG) methods are combined to jointly train multi-channel gains by setting a reasonable segmented reward and punishment mechanism. Through training, a more effective gain decision can be obtained. The robustness of Kalman filtering to interference to a certain extent reduces the precise dependence of reinforcement learning models, thereby achieving higher robustness in intelligent visual servo control. Finally, the effectiveness and advantages of the Kalman-DDPG method have been demonstrated through simulation comparison and six-degree-of-freedom (DOF) uncalibrated manipulator physical experiments.