Myoelectric human computer interaction using CNN-LSTM neural network for dynamic hand gesture recognition

Abstract

Human-computer interaction(HCI) has broad range of applications. One particular application domain is rehabilitation devices. Several bioelectric signals can potentially be used in HCI systems in general and rehabilitation devices in particular. Surface ElectroMyoGraphic(sEMG) signal is one of the more important bioelectric signals in this context. The sEMG signal is formed by muscle activation although the details are rather complex. Applications of sEMG are referred is commonly referred to as myoelectric control since the dominant use of this signal is to activate a device even if (as the term control may imply) feedback is not always used in the process. With the development of deep neural networks, various deep learning architectures are used for sEMG-based gesture recognition with many researchers having reported good performance. Nevertheless, challenges remain in accurately recognizing sEMG patterns generated by gestures produced by hand or the upper arm. For instance one of the difficulties in hand gesture recognition is the influence of limb positions. Several papers have shown that the accuracy of gesture classification decreases when the limb position changes even if the gesture remains the same. Prior work by our team has shown that dynamic gesture recognition is in principle more reliable in detecting human intent, which is often the underlying idea of gesture recognition. In this paper, a Convolutional Neural Network (CNN) with Long Short-Term Memory or LSTM (CNN-LSTM) is proposed to classify five common dynamic gestures. Each dynamic gesture would be performed in five different limb positions as well. The trained neural network model is then used to enable a human subject to control a 6 DoF (Degree of Freedom) robotic arm with 1 DoF gripper. The results show a high level of accurate performance achieved with the proposed approach. In particular, the overall accuracy of the dynamic gesture recognition is 84.2%. The accuracies vary across subjects but remain at approximately 90%for some subjects.