Reinforcement-Learning-Based Visual Servoing of Underwater Vehicle Dual-Manipulator System

Abstract

As a substitute for human arms, underwater vehicle dual-manipulator systems (UVDMSs) have attracted the interest of global researchers. Visual servoing is an important tool for the positioning and tracking control of UVDMSs. In this paper, a reinforcement-learning-based adaptive control strategy for the UVDMS visual servo, considering the model uncertainties, is proposed. Initially, the kinematic control is designed by developing a hybrid visual servo approach using the information from multi-cameras. The command velocity of the whole system is produced through a task priority method. Then, the reinforcement-learning-based velocity tracking control is developed with a dynamic inversion approach. The hybrid visual servoing uses sensors equipped with UVDMSs while requiring fewer image features. Model uncertainties of the coupled nonlinear system are compensated by the actor–critic neural network for better control performances. Moreover, the stability analysis using the Lyapunov theory proves that the system error is ultimately uniformly bounded (UUB). At last, the simulation shows that the proposed control strategy performs well in the task of dynamical positioning.