SFANet: Spatial-Frequency Attention Network for Deepfake Detection

2025-10-06

AI Summary

Deepfake detection models often suffer from poor generalization across diverse datasets and generative techniques. Method: This paper proposes a hybrid detection framework integrating spatial-frequency attention with texture analysis. It innovatively combines facial segmentation and patch-level attention mechanisms, incorporates frequency-domain processing (e.g., DCT decomposition) and local texture modeling, and employs a sequential training strategy to mitigate class imbalance. Architecturally, it unifies Swin Transformer, Vision Transformer (ViT), and an attention fusion network to establish a multimodal feature collaboration representation system. Contribution/Results: Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on the DFWild-Cup multi-source benchmark, significantly improving detection accuracy, cross-domain robustness, and decision interpretability compared to existing approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Detecting manipulated media has now become a pressing issue with the recent rise of deepfakes. Most existing approaches fail to generalize across diverse datasets and generation techniques. We thus propose a novel ensemble framework, combining the strengths of transformer-based architectures, such as Swin Transformers and ViTs, and texture-based methods, to achieve better detection accuracy and robustness. Our method introduces innovative data-splitting, sequential training, frequency splitting, patch-based attention, and face segmentation techniques to handle dataset imbalances, enhance high-impact regions (e.g., eyes and mouth), and improve generalization. Our model achieves state-of-the-art performance when tested on the DFWild-Cup dataset, a diverse subset of eight deepfake datasets. The ensemble benefits from the complementarity of these approaches, with transformers excelling in global feature extraction and texturebased methods providing interpretability. This work demonstrates that hybrid models can effectively address the evolving challenges of deepfake detection, offering a robust solution for real-world applications.

Problem

• Detecting manipulated media across diverse datasets and generation techniques
• Improving detection accuracy and robustness using hybrid ensemble framework
• Handling dataset imbalances while enhancing focus on critical facial regions

Innovation

• Combines transformers and texture methods for detection
• Uses frequency splitting and patch attention techniques
• Implements face segmentation to enhance critical regions