🤖 AI Summary
To address poor cross-dataset generalization and weak adversarial robustness in deepfake detection, this paper proposes a texture-driven dual-branch cross-attention network. Methodologically: (1) we introduce a novel parallel learnable texture enhancement module embedded in the ResNet downsampling front-end to explicitly model both local structures and global textures; (2) we design a dual-branch Vision Transformer (ViT) architecture with cross-branch cross-attention to capture long-range texture inconsistencies; (3) we incorporate multi-domain adversarial training to enhance robustness. Evaluated on cross-domain benchmarks—FF++, DFDCPreview, and Celeb-DF—the method achieves 98% accuracy and demonstrates strong resilience against common post-processing distortions (e.g., blurring, compression, noise) and adversarial attacks. To our knowledge, this is the first work to deeply integrate texture priors with a dual-path ViT architecture, significantly improving both generalization capability and interference resistance.
📝 Abstract
Deepfakes, which employ GAN to produce highly realistic facial modification, are widely regarded as the prevailing method. Traditional CNN have been able to identify bogus media, but they struggle to perform well on different datasets and are vulnerable to adversarial attacks due to their lack of robustness. Vision transformers have demonstrated potential in the realm of image classification problems, but they require enough training data. Motivated by these limitations, this publication introduces Tex-ViT (Texture-Vision Transformer), which enhances CNN features by combining ResNet with a vision transformer. The model combines traditional ResNet features with a texture module that operates in parallel on sections of ResNet before each down-sampling operation. The texture module then serves as an input to the dual branch of the cross-attention vision transformer. It specifically focuses on improving the global texture module, which extracts feature map correlation. Empirical analysis reveals that fake images exhibit smooth textures that do not remain consistent over long distances in manipulations. Experiments were performed on different categories of FF++, such as DF, f2f, FS, and NT, together with other types of GAN datasets in cross-domain scenarios. Furthermore, experiments also conducted on FF++, DFDCPreview, and Celeb-DF dataset underwent several post-processing situations, such as blurring, compression, and noise. The model surpassed the most advanced models in terms of generalization, achieving a 98% accuracy in cross-domain scenarios. This demonstrates its ability to learn the shared distinguishing textural characteristics in the manipulated samples. These experiments provide evidence that the proposed model is capable of being applied to various situations and is resistant to many post-processing procedures.