Can Same-right-and-different-left Gestures Be Recognized with Only Right-hand Signals?

Abstract

Sign language serves as a bridge between the hearing-impaired and other people. Existing sensor-based approaches tend to only collect data from the dominant hand. Does this signal collection method affect the accuracy of gesture recognition, especially gestures where the dominant hand has the same movement while the non-dominant hand has different movements? The specific gestures are called same-right-and-different-left (SRDL) where the right hand is dominant. This article is the first to propose an SRDL-aware sign language recognition system. First, an SRDL discriminator based on an autoencoder and range classifier is designed to determine whether the gesture is SRDL. Second, an SRDL feature selector based on clustering relationship is presented. Multivariate variational mode decomposition and fast fourier transform are used to obtain the feature expression. Moreover, a clustering relationship algorithm is proposed to dynamically select features for every group of SRDL gestures in the feature expression. Finally, the experimental results show that the average word error rate is 14.3% and decreases by 8.5% and 12.1% compared with Signspeaker and MyoSign, respectively.