STAB-GCN: A Spatio-Temporal Attention-Based Graph Convolutional Network for Group Activity Recognition
-
Published:2023-07-12
Issue:14
Volume:11
Page:3074
-
ISSN:2227-7390
-
Container-title:Mathematics
-
language:en
-
Short-container-title:Mathematics
Author:
Liu Fang1ORCID, Tian Chunhua1ORCID, Wang Jinzhong2ORCID, Jin Youwei2, Cui Luxiang3, Lee Ivan4
Affiliation:
1. School of Computer Science, Shenyang Aerospace University, Shenyang 110136, China 2. Public Basic Course Teaching and Research Department, Shenyang Sport University, Shenyang 110102, China 3. Sports Training College, Shenyang Sport University, Shenyang 110102, China 4. STEM, University of South Australia, Mawson Lakes 5095, Australia
Abstract
Group activity recognition is a central theme in many domains, such as sports video analysis, CCTV surveillance, sports tactics, and social scenario understanding. However, there are still challenges in embedding actors’ relations in a multi-person scenario due to occlusion, movement, and light. Current studies mainly focus on collective and individual local features from the spatial and temporal perspectives, which results in inefficiency, low robustness, and low portability. To this end, a Spatio-Temporal Attention-Based Graph Convolution Network (STAB-GCN) model is proposed to effectively embed deep complex relations between actors. Specifically, we leverage the attention mechanism to attentively explore spatio-temporal latent relations between actors. This approach captures spatio-temporal contextual information and improves individual and group embedding. Then, we feed actor relation graphs built from group activity videos into our proposed STAB-GCN for further inference, which selectively attends to the relevant features while ignoring those irrelevant to the relation extraction task. We perform experiments on three available group activity datasets, acquiring better performance than state-of-the-art methods. The results verify the validity of our proposed model and highlight the obstructive impacts of spatio-temporal attention-based graph embedding on group activity recognition.
Funder
Doctoral Research Startup Fund Program of Liaoning Province Scientific Research Fund Project of the Educational Department of Liaoning Province
Subject
General Mathematics,Engineering (miscellaneous),Computer Science (miscellaneous)
Reference40 articles.
1. Han, M.F., Zhang, D.J., Wang, Y.L., Yan, R., Yao, L.N., Chang, X.J., and Qiao, Y. (2022, January 19–24). Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. 2. Wu, J.C., Wang, L.M., Wang, L., Guo, J., and Wu, G.S. (2019, January 16–20). Learning actor relation graphs for group activity recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. 3. Choi, W.G., Shahid, K., and Savarese, S. (October, January 29). What are they doing? Collective activity classification using spatio-temporal relationship among people. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Kyoto, Japan. 4. Spatio-Temporal Graph Learning for Epidemic Prediction;Yu;ACM Trans. Intell. Syst. Technol.,2023 5. Deep Learning for Heterogeneous Human Activity Recognition in Complex IoT Applications;Hawash;IEEE Internet Things J.,2022
|
|