3D Skeletal Volume Templates for Deep Learning-Based Activity Recognition

Abstract

Due to advances in depth sensor technologies, the use of these sensors has positively impacted studies of human-computer interaction and activity recognition. This study proposes a novel 3D action template generated from depth sequence data and two methods to classify single-person activities using this 3D template. Initially, joint skeleton-based three-dimensional volumetric templates are constructed from depth information. In the first method, images are obtained from various view angles of these three-dimensional templates and used for deep feature extraction using a pre-trained convolutional neural network. In our experiments, a pre-trained AlexNet model trained with the ImageNet dataset is used as a feature extractor. Activities are classified by combining deep features and Histogram of Oriented Gradient (HOG) features. The second approach proposes a three-dimensional convolutional neural network that uses volumetric templates as input for activity classification. Proposed methods have been tested with two publicly available datasets. Experiments provided promising results compared with the other studies presented in the literature.