A Quality-Centric Framework for Generic Deepfake Detection

2024-11-08

arXiv.org

AI Summary

To address the limited generalization of deepfake detectors to unseen forgery methods, this paper proposes a forgery-quality-centric curriculum learning framework. Methodologically, it introduces the first forgery-quality-driven curriculum learning paradigm, integrating multimodal quality assessment across frequency and spatial domains; designs a frequency-domain data augmentation strategy specifically for low-quality forged images; and incorporates progressive sampling with adaptive learning-rate scheduling. The framework is plug-and-play—requiring no architectural modifications to the detector backbone. Evaluated on multiple cross-method generalization benchmarks, it significantly outperforms state-of-the-art approaches, achieving an average AUC improvement of 5.2%. This demonstrates substantial gains in robustness against previously unencountered forgery techniques.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Detecting AI-generated images, particularly deepfakes, has become increasingly crucial, with the primary challenge being the generalization to previously unseen manipulation methods. This paper tackles this issue by leveraging the forgery quality of training data to improve the generalization performance of existing deepfake detectors. Generally, the forgery quality of different deepfakes varies: some have easily recognizable forgery clues, while others are highly realistic. Existing works often train detectors on a mix of deepfakes with varying forgery qualities, potentially leading detectors to short-cut the easy-to-spot artifacts from low-quality forgery samples, thereby hurting generalization performance. To tackle this issue, we propose a novel quality-centric framework for generic deepfake detection, which is composed of a Quality Evaluator, a low-quality data enhancement module, and a learning pacing strategy that explicitly incorporates forgery quality into the training process. Our framework is inspired by curriculum learning, which is designed to gradually enable the detector to learn more challenging deepfake samples, starting with easier samples and progressing to more realistic ones. We employ both static and dynamic assessments to assess the forgery quality, combining their scores to produce a final rating for each training sample. The rating score guides the selection of deepfake samples for training, with higher-rated samples having a higher probability of being chosen. Furthermore, we propose a novel frequency data augmentation method specifically designed for low-quality forgery samples, which helps to reduce obvious forgery traces and improve their overall realism. Extensive experiments demonstrate that our proposed framework can be applied plug-and-play to existing detection models and significantly enhance their generalization performance in detection.

Problem

• Improving generalization in deepfake detection across unseen manipulation methods
• Addressing varying forgery quality in training data for better detector performance
• Enhancing low-quality deepfakes to reduce obvious artifacts and improve realism

Innovation

• Quality-centric framework for deepfake detection
• Dynamic and static forgery quality assessment
• Frequency augmentation for low-quality samples