Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion

Abstract

As human–robot interaction and teleoperation technologies advance, anthropomorphic control of humanoid arms has garnered increasing attention. However, accurately translating sensor-detected arm motions to the multi-degree freedom of a humanoid robotic arm is challenging, primarily due to occlusion issues with single-sensor setups, which reduce recognition accuracy. To overcome this problem, we propose a human-like arm control strategy based on multi-sensor fusion. We defined the finger bending angle to represent finger posture and employed a depth camera to capture arm movement. Consequently, we developed an arm movement tracking system and achieved anthropomorphic control of the imitation human arm. Finally, we verified our proposed method’s effectiveness through a series of experiments, evaluating the system’s robustness and real-time performance. The experimental results show that this control strategy can control the motion of the humanoid arm stably, and maintain a high recognition accuracy in the face of complex situations such as occlusion.