Problem
Research questions and friction points this paper is trying to address.
lip-syncing deepfakes
detection
localization
counterfactual consistency
temporal modeling
LoCC: Detection and Localization of Lip-Syncing Deepfakes via Counterfactual Frame Consistency
🤖 AI Summary
Existing methods struggle to effectively detect subtle and dynamically evolving artifacts in the mouth region of lip-sync deepfake videos. To address this challenge, this work proposes the LoCC framework, which introduces— for the first time—a counterfactual frame consistency mechanism within a teacher–student learning paradigm. By generating counterfactual estimates from temporally adjacent frames, LoCC evaluates the spatiotemporal consistency of mouth movements in each frame, enabling fine-grained forgery detection and localization at both clip-level and frame-level granularity. The approach overcomes the limitations of conventional holistic video analysis and achieves state-of-the-art performance across multiple benchmarks, including LAV-DF, AVDF1M, FakeAVCeleb, and KODF. Furthermore, it demonstrates exceptional generalization under varying compression levels and in cross-dataset settings.
📝 Abstract
Lip-syncing deepfakes are among the most challenging forms of manipulated media because their artifacts are localized almost exclusively to the mouth region and evolve dynamically over time. Detecting such deepfakes requires precise temporal and spatial modeling of lip motion. In this paper, we propose LoCC, a novel detection framework that performs fine-grained detection and localization of lip-syncing deepfakes at both segment and frame levels. Unlike prior approaches that analyze videos holistically, our method evaluates whether each frame aligns with a counterfactual estimate generated from its temporal neighbors. Real videos exhibit strong and stable consistency, whereas lip-sync deepfakes introduce localized inconsistencies. Following a teacher-student learning paradigm, our model effectively captures these frame-level discrepancies and achieves superior performance over state-of-the-art methods on multiple benchmark lip-syncing deepfake datasets, including LAV-DF, AVDF1M, FakeAVCeleb, and KODF, and generalizes well across compression levels and datasets.
Innovation
Methods, ideas, or system contributions that make the work stand out.
counterfactual frame consistency
lip-syncing deepfakes
teacher-student learning
temporal modeling
fine-grained localization
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
