Combining Keyframes and Image Classification for Violent Behavior Recognition

Abstract

Surveillance cameras are increasingly prevalent in public places, and security services urgently need to monitor violence in real time. However, the current violent-behavior-recognition models focus on spatiotemporal feature extraction, which has high hardware resource requirements and can be affected by numerous interference factors, such as background information and camera movement. Our experiments have found that violent and non-violent video frames can be classified by deep-learning models. Therefore, this paper proposes a keyframe-based violent-behavior-recognition scheme. Our scheme considers video frames as independent events and judges violent events based on whether the number of keyframes exceeds a given threshold, which reduces hardware requirements. Moreover, to overcome interference factors, we propose a new training method in which the background-removed and original image pair facilitates feature extraction of deep-learning models and does not add any complexity to the networks. Comprehensive experiments demonstrate that our scheme achieves state-of-the-art performance for the RLVS, Violent Flow, and Hockey Fights datasets, outperforming existing methods.