Intelligent Recognition of Smoking and Calling Behaviors for Safety Surveillance

Abstract

Smoking and calling are two typical behaviors involved in public and industrial safety that usually need to be strictly monitored and even prohibited on many occasions. To resolve the problems of missed detection and false detection in the existing traditional and deep-learning-based behavior-recognition methods, an intelligent recognition method using a multi-task YOLOv4 (MT-YOLOv4) network combined with behavioral priors is proposed. The original YOLOv4 is taken as the baseline network to be improved in the proposed method. Firstly, a K-means++ algorithm is used to re-cluster and optimize the anchor boxes, which are a set of predefined bounding boxes to capture the scale and aspect ratio of specific objects. Then, the network is divided into two branches with the same blocks but independent tasks after the shared feature extraction layer of CSPDarknet-53, i.e., the behavior-detection branch and the object-detection branch, which predict the behaviors and their related objects respectively from the input image or video frame. Finally, according to the preliminary predicted results of the two branches, comprehensive reasoning rules are established to obtain the final behavior-recognition result. A dataset on smoking and calling detection is constructed for training and testing, and the experimental results indicate that the proposed method has a 6.2% improvement in recall and a 2.4% improvement in F1 score at the cost of a slight loss in precision compared to the baseline method; the proposed method achieved the best performance among the compared methods. It can be deployed to related security surveillance systems for unsafe-behavior monitoring and early-warning management in practical scenarios.