MyoTac: Real-Time Recognition of Tactical Sign Language Based on Lightweight Deep Neural Network

Abstract

Real-time tactical sign language recognition enables communication in a silent environment and outside the visual range, and human-computer interaction (HCI) can also be realized. Although the existing methods have high accuracy, they cannot be conveniently implemented in a portable system due to the complexity of their models. In this paper, we present MyoTac, a user-independent real-time tactical sign language classification system that makes the network lightweight through knowledge distillation, so as to balance between high accuracy and execution efficiency. We design tactical convolutional neural networks (TCNN) and bidirectional long short-term memory (B-LSTM) to capture the spatial and temporal features of the signals, respectively, and extract the soft target with knowledge distillation to compress the scale of the neural network by nearly four times without affecting the accuracy. We evaluate MyoTac on 30 tactical sign language (TSL) words based on data from 38 volunteers, including 25 volunteers collecting offline data and 13 volunteers conducting online tests. When dealing with new users, MyoTac achieves an average accuracy of 92.67% and the average recognition time is 2.81 ms. The obtained results show that our approach outperforms other algorithms proposed in the literature, reducing the real-time recognition time by 84.4% with higher accuracy.