OTRN-DCN: An optimized transformer-based residual network with deep convolutional network for action recognition and multi-object tracking of adaptive segmentation using soccer sports video

Abstract

In today’s era, video analysis is immensely involved in recognizing the sport-related movement that has become a significant part of human’s life. The intent of this approach is to know about the player’s activities with prior information of tracking objects. It also analyzes the player potential or capacity to lead the winning team. When the player frequently changes their location, object tracking and action recognition will become a quite challenging task. Over the game, various athletes or different objects are considered to assist the system to easily recognize the respective actions of the player. Most of the previous models have been implemented, yet, it faces such consequences to provide promising performance. To meet the pre-requisite, a new multi-athlete tracking model for action recognition in soccer sports is designed with deep learning approaches. Initially, the multi-object tracking video is offered as the input to pre-processing phase. Here, occlusion and background clutter removal and contrast enhancement techniques are utilized to perform pre-processing in the videos. Then, the pre-processed video is offered to the multi-object tracking phase, where the jersey number is observed during multi-object tracking to avoid the identity switch problem. Then, effective multi-object tracking is performed by adaptive YOLOv5. The parameters presented in the improved adaptive YOLOv5 are tuned by proposing a new algorithm as the Random-based Cheetah Red Deer Algorithm (RCRDA). Next, in the action recognition phase, the tracked object from the video is taken based on the Region of Interest (ROI) that is subjected to an action recognition model named Optimized Transformer-based Residual Network with Deep Convolutional Network (OTRN-DCN). At first, ROI is offered as the input to TRN for attaining the feature vectors. Then, the optimal weighted vector extraction is performed, where the weight is tuned by the developed RCRDA. Finally, the attained optimally weighted vectors are given to the DCN phase for attaining recognized action as output. Hence, the developed multi-object tracking and action recognition model will secure an improved recognition rate than the traditional framework.