Hybrid Deepfake Image Detection: A Comprehensive Dataset-Driven Approach Integrating Convolutional and Attention Mechanisms with Frequency Domain Features

2025-02-15

AI Summary

To address the challenge of cross-generator generalization in deepfake image detection, this paper proposes a multi-model collaborative detection framework integrating local texture, global semantics, and frequency-domain artifacts. We innovatively design three heterogeneous architectures—ResNet-34, DeiT, and a wavelet-enhanced XceptionNet—and fuse their predictions via weighted ensemble learning, augmented by interpretability-driven optimization: Grad-CAM identifies discriminative regions, while t-SNE validates inter-class feature separability. Evaluated on the multi-source DFWild-Cup 2025 benchmark, the ensemble achieves 93.23% accuracy and 97.44% AUC on the SP Cup 2025 validation set—substantially outperforming individual models (best: 89.32% for DeiT) and demonstrating strong class-wise stability per confusion matrix analysis. To our knowledge, this is the first work to synergistically integrate frequency-domain enhancement with vision transformers in deepfake detection, delivering simultaneous advances in both detection performance and model interpretability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Effective deepfake detection tools are becoming increasingly essential over the last few years due to the growing usage of deepfakes in unethical practices. There exists a diverse range of deepfake generation techniques, which makes it challenging to develop an accurate universal detection mechanism. The 2025 Signal Processing Cup (DFWild-Cup competition) provided a diverse dataset of deepfake images, which are generated from multiple deepfake image generators, for training machine learning model(s) to emphasize the generalization of deepfake detection. To this end, we proposed an ensemble-based approach that employs three different neural network architectures: a ResNet-34-based architecture, a data-efficient image transformer (DeiT), and an XceptionNet with Wavelet Transform to capture both local and global features of deepfakes. We visualize the specific regions that these models focus for classification using Grad-CAM, and empirically demonstrate the effectiveness of these models in grouping real and fake images into cohesive clusters using t-SNE plots. Individually, the ResNet-34 architecture has achieved 88.9% accuracy, whereas the Xception network and the DeiT architecture have achieved 87.76% and 89.32% accuracy, respectively. With these networks, our weighted ensemble model achieves an excellent accuracy of 93.23% on the validation dataset of the SP Cup 2025 competition. Finally, the confusion matrix and an Area Under the ROC curve of 97.44% further confirm the stability of our proposed method.

Problem

• Hybrid deepfake detection method
• Combines convolutional and attention mechanisms
• Integrates frequency domain features

Innovation

• Combines convolutional and attention mechanisms
• Integrates frequency domain features
• Uses ensemble-based approach with three architectures