Cross-attention based two-branch networks for document image forgery localization in the Metaverse

Abstract

In recent years, the Metaverse has garnered significant attention in social and Metahuman realms, showcasing substantial value and immense developmental potential through its integration of virtual and real worlds. However, this integration has also raised security concerns. For instance, in digital image forensics, the malicious dissemination of false images by wrongdoers could result in serious consequences and the propagation of misinformation. This paper presented a novel two-branch network (abbreviated as CAFTB-Net) to detect and localize the forged regions of document images in the Metaverse. One branch extracts manipulation trace directly from spatial information, e.g., unnatural smears, anomalies between pixels, etc. The other branch employs an SRM filter to transform the input image from the color domain into the noise domain, effectively extracting anomalies, such as global noise inconsistencies from the noise domain. Compared to spatial domain features, the discontinuity of forgery traces within the noise domain aids the network in authenticating the document image. The two branches extract local and global features in the forged document images. Finally, we propose a cross-attention module to fuse local spatial features and global noise features. Extensive experimental results demonstrate that the proposed network achieves F1 scores of 0.819 and 0.948 on the SACP and ICDAR datasets, respectively, with AUC scores of 0.933 and 0.764, outperforming some of the state-of-the-art algorithms.