Spatiotemporal Detection and Localization of Object Removal Video Forgery with Multiple Feature Extraction and Optimized Residual Network

Abstract

Video forgery detection and localization is one of the most important issue due to the advanced editing software that provides strengthen to tools for manipulating the videos. Object based video tampering destroys the originality of the video. The main aim of the video forensic is to eradicate the forgeries from the original video that are useful in various applications. However, the research on detecting and localizing the object based video forgery with advanced techniques still remains the open and challenging issue. Many of the existing techniques have focused only on detecting the forged video under static background that cannot be applicable for detecting the forgery in tampered video. In addition to this, conventional techniques fail to extract the essential features in order to investigate the depth of the video forgery. Hence, this paper brings a novel technique for detecting and localizing the forged video with multiple features. The steps involved in this research are keyframe extraction, pre-processing, feature extraction and finally detection and localization of forged video. Initially, keyframe extraction uses the Gaussian mixture model (GMM) to extract frames from the forged videos. Then, the pre-processing stage is manipulated to convert the RGB frame into a grayscale image. Multi-features need to be extracted from the pre-processed frames to study the nature of the forged videos. In our proposed study, speeded up robust features (SURF), principal compound analysis histogram oriented gradients (PCA-HOG), model based fast digit feature (MBFDF), correlation of adjacent frames (CAF), the prediction residual gradient (PRG) and optical flow gradient (OFG) features are extracted. The dataset used for the proposed approach is collected from REWIND of about 40 forged and 40 authenticated videos. With the help of the DL approach, video forgery can be detected and localized. Thus, this research mainly focuses on detecting and localization of forged video based on the ResNet152V2 model hybrid with the bidirectional gated recurrent unit (Bi-GRU) to attain maximum accuracy and efficiency. The performance of this approach is finally compared with existing approaches in terms of accuracy, precision, F-measure, sensitivity, specificity, false-negative rate (FNR), false discovery rate (FDR), false-positive rate (FPR), Mathew’s correlation coefficient (MCC) and negative predictive value (NPV). The proposed approach assures the performance of 96.17% accuracy, 96% precision, 96.14% F-measure, 96.58% sensitivity, 96.5% specificity, 0.034 FNR, 0.04 FDR, 0.034 FPR, 0.92 MCC and 96% NPV, respectively. Along with is, the mean square error (MSE) and peak-to-signal-noise ratio (PSNR) for the GMM model attained about 104 and 27.95, respectively.