Author:
Yang Huaigang,Ren Ziliang,Yuan Huaqiang,Wei Wenhong,Zhang Qieshi,Zhang Zhaolong
Abstract
Graph convolution networks (GCNs) have been widely used in the field of skeleton-based human action recognition. However, it is still difficult to improve recognition performance and reduce parameter complexity. In this paper, a novel multi-scale attention spatiotemporal GCN (MSA-STGCN) is proposed for human violence action recognition by learning spatiotemporal features from four different skeleton modality variants. Firstly, the original joint data are preprocessed to obtain joint position, bone vector, joint motion and bone motion datas as inputs of recognition framework. Then, a spatial multi-scale graph convolution network based on the attention mechanism is constructed to obtain the spatial features from joint nodes, while a temporal graph convolution network in the form of hybrid dilation convolution is designed to enlarge the receptive field of the feature map and capture multi-scale context information. Finally, the specific relationship in the different skeleton data is explored by fusing the information of multi-stream related to human joints and bones. To evaluate the performance of the proposed MSA-STGCN, a skeleton violence action dataset: Filtered NTU RGB+D was constructed based on NTU RGB+D120. We conducted experiments on constructed Filtered NTU RGB+D and Kinetics Skeleton 400 datasets to verify the performance of the proposed recognition framework. The proposed method achieves an accuracy of 95.3% on the Filtered NTU RGB+D with the parameters 1.21M, and an accuracy of 36.2% (Top-1) and 58.5% (Top-5) on the Kinetics Skeleton 400, respectively. The experimental results on these two skeleton datasets show that the proposed recognition framework can effectively recognize violence actions without adding parameters.
Subject
Artificial Intelligence,Biomedical Engineering
Reference48 articles.
1. Skeleton image representation for 3D action recognition based on tree structure and reference joints,;Caetano
2. Skelemotion: a new representation of skeleton joint sequences based on motion information for 3D action recognition,;Caetano
3. Body joint guided 3-d deep convolutional descriptors for action recognition;Cao;IEEE Trans. Cybern,2018
4. Openpose: realtime multi-person 2D pose estimation using part affinity fields;Cao;IEEE Trans. Pattern Anal. Mach. Intell,2021
5. Quo vadis, action recognition? a new model and the kinetics dataset,;Carreira,2017