Wearable Hand Pose Estimation for Remote Control of a Robot on the Moon

Abstract

In recent years, a plan has been developed to conduct an investigation of an unknown environment by remotely controlling an exploration robot system sent to the Moon. For the robot to successfully perform sophisticated tasks, it must implement multiple degrees of freedom in not only its arms and legs but also its hands and fingers. Moreover, the robot has to be a humanoid type so that it can use tools used by astronauts, with no modification. On the other hand, to manipulate the multiple-degrees-of-freedom robot without learning skills necessary for manipulation and to minimize the psychological burden on operators, employing a method that utilizes everyday movements of operators as input to the robot, rather than a special controller, is ideal. In this paper, the authors propose a compact wearable device that allows for the estimation of the hand pose (hand motion capture) of the subject. The device has a miniature wireless RGB camera that is placed on the back of the user’s hand, rather than on the palm. Attachment of the small camera to the back of the hand may make it possible to minimize the restraint on the subject’s motions during motion capture. In the conventional techniques, the camera is attached on the palm because images of fingertips always need to be captured by the camera before images of any other part of the hand can be captured. In contrast, the image processing algorithm proposed herein is capable of estimating the hand pose with no need for first capturing the fingertips.