TCSD: Triple Complementary Streams Detector for Comprehensive Deepfake Detection

Abstract

Advancements in computer vision and deep learning have made it difficult to distinguish generated Deepfake media in visual. While existing detection frameworks have achieved significant performance on the challenging Deepfake datasets, these approaches consider a single perspective. More importantly, in urban scenes, neither complex scenarios can be covered by a single view, nor the correlation between multiple information is well utilized. In this paper, to mine the new view for Deepfake detection and utilize the correlation of multi-view information contained in images, we propose a novel triple complementary streams detector, namely TCSD. Specifically, firstly, a novel depth estimator is designed to perceive depth information (DI) which is not utilized by previous methods. Then, to supplement the depth information for obtaining comprehensive forgery clues, we consider the incoherence between image foreground and background information (FBI) and the inconsistency between local and global information (LGI). In addition, attention-based multi-scale feature extraction (MsFE) module is designed to perceive more complementary features from DI, FBI and LGI. Finally, two attention-based feature fusion modules are proposed to adaptively fuse information. Extensive experiment results show that the proposed approach achieves the state-of-the-art performance on detecting Deepfake.