Accurate and Accessible Motion-Capture Glove Calibration for Sign Language Data Collection

Abstract

Motion-capture recordings of sign language are used in research on automatic recognition of sign language or generation of sign language animations, which have accessibility applications for deaf users with low levels of written-language literacy. Motion-capture gloves are used to record the wearer’s handshape. Unfortunately, they require a time-consuming and inexact calibration process each time they are worn. This article describes the design and evaluation of a new calibration protocol for motion-capture gloves, which is designed to make the process more efficient and to be accessible for participants who are deaf and use American Sign Language (ASL). The protocol was evaluated experimentally; deaf ASL signers wore the gloves, were calibrated (using the new protocol and using a calibration routine provided by the glove manufacturer), and were asked to perform sequences of ASL handshapes. Five native ASL signers rated the correctness and understandability of the collected handshape data. In an additional evaluation, ASL signers were asked to perform ASL stories while wearing the gloves and a motion-capture bodysuit (in some cases our new calibration protocol was used, in other cases, the standard protocol). Later, twelve native ASL signers watched animations produced from this motion-capture data and answered comprehension questions about the stories. In both evaluation studies, the new protocol received significantly higher scores than the standard calibration. The protocol has been made freely available online, and it includes directions for the researcher, images and videos of how participants move their hands during the process, and directions for participants (as ASL videos and English text).