Deep learning based spatio‐temporal hand gesture recognition system in complex environment

Abstract

AbstractGesture recognition nowadays has grabbed the attention of researchers as they represent human behaviour in multiple practical ways. Amongst a variety of gestures available, hand gestures play an essential role in the field of human‐computer interaction when recognised efficiently in complex and dynamic environments. In this paper, we propose a dynamic hand gesture recognition system to recognise hand gestures appearing in different indoor and outdoor environments. Hand detection and tracking uses a two‐level system resulting in the formation of gesture trajectory in challenging conditions in which existing detection and tracking algorithms could not do so. A set of 45 features is provided as input to the various classification techniques. The redundancy problem has been reduced by selecting a set of optimum features using the analysis of variance method, which ranks the list of features. An incremental feature selection technique calculates recognition accuracy by selecting features according to rankings. This system provides an accuracy of 96.32% when used with machine learning and 97.5% when used with deep learning techniques. Recognition accuracy is calculated for various environments, including an extra hand, multiple persons in the video frame, and outdoor environment. All machine‐learning classifiers are combined using classifier combination to calculate the accuracy according to the majority‐voting rule. Based on the experimental results, it has been observed that deep learning provides better results compared to machine learning.