Micro-network Based Convolutional Neural Network with Integration of Multilayer Feature Fusion Strategy for Human Activity Recognition

Abstract

Convolutional neural networks (CNN) have shown remarkable performance in enormous computer vision applications over the years, and many works have been done for human activity recognition (HAR) using CNN. However, most of deep learning-based architectures require large training data and plenty of computational resources. Therefore, we proposed a simple and efficient deep learning model for human activity recognition, which works on complex visual data and require less computational resources. In the proposed work, we designed a novel CNN architecture by stacking the repeated components called micro-networks to incorporate multiscale processing in the network layers. We have used a feature fusion strategy to pass the previous layer’s abstract complementary information to the next adjacent layer, believing that each layer encapsulates specific feature maps. Therefore, the hidden complementary information can potentially enhance the feature discrimination capacity of the network and help in learning the network. The proposed architecture is fully trained from scratch with stochastic gradient descent (SGD) optimizer at 0.05 initial learning rate and a softmax classifier is used for activity recognition. The merit of the proposed method over standard deep learning models is its computational efficiency in terms of learnable parameters and computational resources. The proposed model gives good performance on small as well as large size datasets. For authentication of the proposed method several extensive experiments are conducted on publically available datasets, namely UCF-101, HMDB-51, YouTube, and IXMAS datasets. The results have shown the outperformance of the proposed method over the existing state-of-the-art methods.