🤖 AI Summary
Deepfake videos have become increasingly photorealistic, rendering them indistinguishable to the human eye, while existing detection models lack effective human-in-the-loop mechanisms. To address this, we propose a human-AI collaborative detection enhancement framework. First, we design a semi-supervised Artifact Attention module that leverages limited human feedback to generate forgery artifact attention maps. Second, we introduce “Deepfake Caricatures”—an interpretable visual enhancement technique that amplifies salient artifacts to jointly improve human perceptual sensitivity and model discriminability. The method integrates semi-supervised learning, attention mechanisms, and human perception modeling. Experiments demonstrate significant improvements across diverse temporal lengths and user engagement levels: human detection accuracy increases by +12.3%, and model performance (AUC) improves by +4.7%. A user study confirms dual breakthroughs in both human and AI detection capabilities, validating the efficacy of our collaborative paradigm.
📝 Abstract
Deepfakes pose a serious threat to digital well-being by fueling misinformation. As deepfakes get harder to recognize with the naked eye, human users become increasingly reliant on deepfake detection models to decide if a video is real or fake. Currently, models yield a prediction for a video's authenticity, but do not integrate a method for alerting a human user. We introduce a framework for amplifying artifacts in deepfake videos to make them more detectable by people. We propose a novel, semi-supervised Artifact Attention module, which is trained on human responses to create attention maps that highlight video artifacts. These maps make two contributions. First, they improve the performance of our deepfake detection classifier. Second, they allow us to generate novel"Deepfake Caricatures": transformations of the deepfake that exacerbate artifacts to improve human detection. In a user study, we demonstrate that Caricatures greatly increase human detection, across video presentation times and user engagement levels. Overall, we demonstrate the success of a human-centered approach to designing deepfake mitigation methods.
