HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales

2026-07-09

AI Summary

Existing deepfake video benchmarks inadequately model human-object and human-human interactions as well as multimodal alignment, rendering them ill-equipped to handle the high-fidelity, human-centric synthetic videos generated by diffusion models. To address this gap, this work introduces HumanForge, a large-scale, multi-paradigm dataset of human-centric forged videos, along with Gen2Anno, a multi-agent active annotation framework. Built upon a LangGraph-based six-agent collaborative architecture, Gen2Anno integrates mixture-of-experts reference analysis, closed-loop forensic validation, video diffusion models, and multimodal large language models to autonomously generate over 18K high-fidelity video clips paired with structured annotations—including binary authenticity labels, fine-grained artifact categories, and spatiotemporal localization. Benchmark evaluations reveal that current detection methods exhibit significant limitations in zero-shot generalization and fine-grained reasoning.

Abstract

Rapid advancements in video diffusion models and temporal editing tools have enabled the generation of highly realistic human-centric videos, posing unprecedented challenges to digital content forensics. Existing benchmarks primarily focus on either face-swapping or global text-to-video synthesis, overlooking the crucial dimensions of human-object or human-human interactions and multi-modal alignment. To address these limitations, we introduce HumanForge, a unified, large-scale, and multi-paradigm human-centric video forgery dataset. To construct and annotate this dataset without labor-intensive manual labeling or hallucinated monolithic prompts, we propose Gen2Anno, a modular active multi-agent pipeline built on LangGraph. Gen2Anno coordinates six specialized agents-ranging from source profiling to MoE-based reference analysis and closed-loop forensic verification-to generate over 18K high-fidelity video segments and produce structured, contrastive omni-annotations containing binary decisions, fine-grained artifact categories, and spatio-temporal localization. Extensive benchmarks using state-of-the-art traditional detectors and Large Multimodal Models (LMMs) demonstrate the significant challenges of zero-shot generalization and fine-grained reasoning on HumanForge. Code and dataset will be publicly released.

Problem

• deepfake video
• human-centric interaction
• multimodal alignment
• digital forensics
• video forgery benchmark

Innovation

• multi-agent annotation
• human-centric deepfake
• video forgery benchmark
• Gen2Anno pipeline
• structured omni-annotation