GAN-based Style Transformation to Improve Gesture-recognition Accuracy

Abstract

Gesture recognition and human-activity recognition from multi-channel sensory data are important tasks in wearable and ubiquitous computing. In these tasks, increasing both the number of recognizable activity classes and the recognition accuracy is essential. However, this is usually an ill-posed problem because individual differences in the same gesture class may affect the discrimination of different gesture classes. One promising solution is to use personal classifiers, but this requires personal gesture samples for re-training the classifiers. We propose a method of solving this issue that obtains personal gesture classifiers using few user gesture samples, thus, achieving accurate gesture recognition for an increased number of gesture classes without requiring extensive user calibration. The novelty of our method is introducing a generative adversarial network (GAN)-based style transformer to 'generate' a user's gesture data. The method synthesizes the gesture examples of the target class of a target user by transforming of a) gesture data into another class of the same user (intra-user transformation) or b) gesture data of the same class of another user (inter-user transformation). The synthesized data are then used to train the personal gesture classifier. We conducted comprehensive experiments using 1) different classifiers including SVM and CNN, 2) intra- and inter-user transformations, 3) various data-missing patterns, and 4) two different types of sensory data. Results showed that the proposed method had an increased performance. Specially, the CNN-based classifiers increased in average accuracy from 0.747 to 0.822 in the CheekInput dataset and from 0.856 to 0.899 in the USC-HAD dataset. Moreover, the experimental results with various data-missing conditions revealed a relation between the number of missing gesture classes and the accuracy of the existing and proposed methods, and we were able to clarify several advantages of the proposed method. These results indicate the potential of considerably reducing the number of required training samples of target users.