Object detection and classification from compressed video streams

Abstract

AbstractVideo Analytics is widely used by the internet‐based platforms to govern the mass consumption of videos. Traditionally, it is carried out from the decoded format of the videos. This requires the analytics server to perform both decoding and analytics computation. This process can be made fast and efficient if performed over the compressed format of the videos as it reduces the decoding stress over the analytics server. The field of video analytics from the binarized formats using modern deep learning techniques is still emerging and needs further exploration. This proposed work is based on the same notion. In this work, two analytics tasks that is, classification and object detection are carried out from the binarized videos. The binarized formats are produced by using an already‐designed end‐to‐end video compression network. The experiments have been carried out over standard datasets. The proposed MobileNetv2‐based classification network shows an accuracy of 66% over the YouTube UGC dataset and the YOLOX‐S‐based detection network shows mAP of 45% over IMAGENet datasets. The proposed work shows competitiveness and improvement in the detection outcomes on compressed data and also provides further motivation for the adoption of deep learning‐based video compression in practical analytics domains.