From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes

2025-04-07

AI Summary

Deepfake detection methods often suffer from poor generalization across unseen generative models. Method: This paper proposes a unified detection framework based on universal artifact modeling. It introduces, for the first time, a principled dichotomy of facial inconsistency artifacts (FIA) and upsampling artifacts (USA), moving away from generator-specific traces. At the data level, synthetic fake samples are generated by injecting only FIA—modeled via an image-level self-blending Blender module—and USA—modeled via super-resolution. A standard image classifier is then trained end-to-end on this synthetic data. Results: The resulting model, trained solely on these synthetically generated samples, achieves significantly improved cross-generator generalization on unseen deepfake datasets. Extensive experiments validate the effectiveness and practicality of the “universal-artifact-driven” paradigm, demonstrating robust detection performance without requiring access to real fake data or knowledge of target generators.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques. In this paper, we ask: How can we build a universal detection framework that is effective for most facial deepfakes? One significant challenge is the wide variety of deepfake generators available, resulting in varying forgery artifacts (e.g., lighting inconsistency, color mismatch, etc). But should we ``teach"the detector to learn all these artifacts separately? It is impossible and impractical to elaborate on them all. So the core idea is to pinpoint the more common and general artifacts across different deepfakes. Accordingly, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). FIA arise from the challenge of generating all intricate details, inevitably causing inconsistencies between the complex facial features and relatively uniform surrounding areas. USA, on the other hand, are the inevitable traces left by the generator's decoder during the up-sampling process. This categorization stems from the observation that all existing deepfakes typically exhibit one or both of these artifacts. To achieve this, we propose a new data-level pseudo-fake creation framework that constructs fake samples with only the FIA and USA, without introducing extra less-general artifacts. Specifically, we employ a super-resolution to simulate the USA, while design a Blender module that uses image-level self-blending on diverse facial regions to create the FIA. We surprisingly found that, with this intuitive design, a standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to unseen deepfakes.

Problem

• Develop universal framework for detecting diverse facial deepfakes
• Identify general artifacts (FIA and USA) across deepfake types
• Create pseudo-fake data to train generalizable detection models

Innovation

• Categorizes deepfake artifacts into FIA and USA
• Uses super-resolution to simulate up-sampling artifacts
• Employs Blender module for face inconsistency artifacts