Three-Dimensional Input System Employing Pinching Gestures for Robot Design

Abstract

Several studies of input interfaces capable of recognizing the gestures have been conducted but most of them use the user’s fingers to enter the position data. These finger-based input interfaces are difficult to provide a so-called click & drag function (as in a mouse) and some of them request for the user to take uncomfortable gestures. When people pinch any objects, however, basically their thumb and index finger come into contact with each other or separate them from each other. These pinching gestures provide superior benefits as the gestures, which may contribute to the input interfaces. This study proposes the method for detecting 3D finger positions and estimating 3D hand postures in pinching gestures based on information on depth images captured by a depth sensor, especially from the viewpoint of robot design. That produces benefits including button-clicking-like input operation by means of contact between the fingers; user’s comfortable gestures as in daily life; clicking action independent of input of positions and postures; and clear identification between ON and OFF. As the evaluation of the 3D input interface proposed here, the authors design real products with the system and a 3D printer, suggesting that the users can design precise and fine 3D objects with his/her comfortable daily gestures with highest usability.