Two Stream Convolution Fusion Network based on Attention Mechanism

Abstract

Abstract Due to the intuitiveness and low cost of image data, action recognition based on image data has always been the focus of action recognition research. The most of studies show that spatio-temporal two-stream network has very superior performance in the field of action recognition. However, traditional two-stream convolutional model needs multiple training, and the method of fusion two-stream models is too simple. In this research, a new two stream convolution fusion model is proposed, which can train our model end-to-end. At the same time, the basic model of spatio-temporal flow model is replaced by CBAM_Resnet. The model is improved on the basis of resnet_101, which gives weight to channel and spatial features, so as to improve the performance of the model. In this paper, the RGB image and the optical flow in X and Y directions are taken as the input features of the fusion network, and the final classification result of the model is obtained. Our method is validated on HMDB51 datasets, and the experimental results show that the accuracy of our model is improved by 4%.