FORTE: Few Samples for Recognizing Hand Gestures with a Smartphone-attached Radar

Abstract

Radar sensing technologies offer several advantages over other gesture input modalities, such as the ability to reliably sense human movements, a reasonable deployment cost, insensitivity to ambient conditions such as light, temperature, and the ability to preserve anonymity. These advantages come at the price of high processing complexity mainly due to the spatio-temporal variations of gesture articulation performed by different people. Deep learning methods, such as CNN-LSTM and 3D CNN-LSTM, have a high potential to recognize radar-based gestures but usually require hundreds or thousands of labeled training samples and high processing power. Asking a lot of people to acquire a lot of gestures is particularly tedious and tiring to the point of being unrealistic. To overcome these challenges, we propose FORTE, a hand gesture recognition with few samples based on an optimized CNN architecture working on pre-processed raw data. Using a k=5-fold cross-validation, we define and compare three alternative CNNs for recognizing hand gestures acquired in a semi-mobile context of use with a portable radar attached to a smartphone. The best CNN reaches an accuracy of 94.96% with a precision of 95.92% and a recall of 96.03% for a dataset composed of solely 5 participants producing 2 samples for 20 classes covering 1 pointing, 2 pantomimic, 3 iconic, and 14 semaphoric gestures. We suggest some implications for designing radar-based gestures and we discuss the limitations of this approach.