Real-time Arm Gesture Recognition in Smart Home Scenarios via Millimeter Wave Sensing

Abstract

"In air" gesture recognition using millimeter wave (mmWave) radar and its applications in natural human-computer-interaction for smart home has shown its potential. However, the state-of-the-art works still fall short in terms of limited gesture space, vulnerable to surrounding interference, and off-line recognition. In this paper, we propose mHomeGes, a real-time mmWave arm gesture recognition system for practical smart home-usage. To this end, we first distill arm gesture's position and dynamic variation, and then custom-design a lightweight convolution neural network to recognize fine-grained gestures. Next, we propose a user discovery method to focus on the target human gesture, thus eliminating the adverse impact of surrounding interference. Finally, we design a hidden Markov model-based voting mechanism to handle continuous gesture signals at run-time, which leads to continuous gesture recognition in real-time. We implement mHomeGes on a commodity mmWave radar and also perform a user study, which demonstrates that mHomeGes achieves high recognition accuracy above 95.30% in real-time across various smart home scenarios, regardless of the impact of surrounding movements, concurrent gestures, human physiological conditions, and outer packing materials. Moreover, we have also publicly archived a mmWave gesture data-set collected during developing mHomeGes, which consists of about 22,000 instances from 25 persons and may have an independent value of facilitating future research.