Towards Generalizable Deepfake Image Detection with Vision Transformers

2026-04-19

AI Summary

This work addresses the limited generalization of existing deepfake detection methods in the face of rapidly evolving generative models and diverse forgery techniques. It proposes a robust detection system by integrating fine-tuned state-of-the-art vision Transformer models—DINOv2, AIMv2, and OpenCLIP ViT-L/14—and optimizes them on the large-scale in-the-wild dataset DF-Wild. Evaluated on the DF-Wild test set, the proposed approach achieves an AUC of 96.77% and an EER of 9%, outperforming the current best method by 7.05% in AUC and 8% in EER. This significant improvement in detecting unseen forgery types earned the method first place in the IEEE S&P Cup 2025.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

In today's day and age, we face a challenge in detecting deepfake images because of the fast evolution of modern generative models and the poor generalization capability of existing methods. In this paper, we use an ensemble of fine-tuned vision transformers like DINOv2, AIMv2 and OpenCLIP's ViT-L/14 to create generalizable method to detect deepfakes. We use the DF-Wild dataset released as part of the IEEE SP Cup 2025, because it uses a challenging and diverse set of manipulations and generation techniques. We started our experiments with CNN classifiers trained on spatial features. Experimental results show that our ensemble outperforms individual models and strong CNN baselines, achieving an AUC of 96.77% and an Equal Error Rate (EER) of just 9% on the DF-Wild test set, beating the state-of-the-art deepfake detection algorithm Effort by 7.05% and 8% in AUC and EER respectively. This was the winning solution for SP Cup, presented at ICASSP 2025.

Problem

• deepfake detection
• generalization
• vision transformers
• generative models
• image manipulation

Innovation

• Vision Transformers
• Deepfake Detection
• Model Ensemble
• Generalization
• DF-Wild Dataset