Seq2seq model for human action recognition based on skeleton and two-layer bidirectional LSTM

Abstract

Human action recognition (HAR) plays an important role in social interaction in various fields. This study proposes a light-weight skeleton and two-layer bidirectional LSTM-based Seq2Seq model (SB2_Seq2Seq) for HAR to trade off recognition accuracy, users’ privacy and computer resource usage. An experiment was conducted to compare the proposed SB2_Seq2Seq with other skeleton-based Seq2Seq models and non-skeleton RGB video frame-based LSTM, CNN and seq2seq models. The UCF50 dataset was used for model evaluation, where 60%, 20% and 20% for model training, validation and testing, respectively. The experimental results show that the proposed model achieves 93.54% accuracy with 0.0214 Mean Square Error (MSE), suggesting that the proposed model outperforms all the other models. Besides, it also shows that the proposed model achieves state-of-the-art accuracy compared with state-of-the-arts methods in literature.