Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination

2026-08-14

AI Summary

This study addresses the absence of benchmarks and insufficient generalization in AI video detection for crisis scenarios by introducing RA-Bench, the first benchmark anchored in real-world videos. Through a multidimensional evaluation framework encompassing large-scale datasets, multimodal zero-shot detection, fine-tuned MLLMs, and social propagation simulation, this work systematically assesses detector performance in authentic crisis environments. Findings reveal that three categories of existing detectors fail to generalize robustly, with generative attributes significantly impacting detection efficacy and social dissemination further compromising reliability. By exposing critical limitations in current approaches to identifying realistic AI-generated videos, this research provides essential empirical evidence to guide the development of more robust detection technologies.

Abstract

Recent video generators can fabricate realistic depictions of wars, disasters, public emergencies, and other real-world crises, creating substantial risks of misinformation. Existing benchmarks, however, provide limited evidence on detector and generator behavior in such settings, including how detectability varies with generation conditions, how people perceive generated videos, and whether detectors remain reliable during social dissemination. To address this gap, we introduce RA-Bench, a benchmark for AI-generated video detection that uses Real videos as Anchors. RA-Bench contains 17,886 videos, comprising 1,830 real-video anchors across 10 social-risk categories and 16,056 generated clips from four open-source and five closed-source generators. Based on RA-Bench, we organize our evaluation along three dimensions. We first assess detector generalization across seven traditional detectors, ten zero-shot multimodal models under three review settings, and two MLLMs specifically fine-tuned on AI-generated video detection. Across these methods, none of the three detector families generalizes consistently across RA-Bench instances. We then examine how detectability varies with generation quality, conditioning information, and sampling seeds. These analyses show that generation properties affect detector families differently, while source-level detection patterns remain stable across seeds. Finally, we study human authenticity judgments and detector reliability during social dissemination. We find that videos that mislead people are also difficult for current detectors, and that social dissemination makes detection harder. Together, these findings show that current methods struggle to detect realistic AI-generated videos, highlighting the need for detectors robust to evolving video generators.

Problem

• AI-generated video detection
• real-world crisis events
• misinformation
• detector generalization
• social dissemination

Innovation

• RA-Bench
• AI-generated video detection
• Real-world crisis events
• Social dissemination
• Systematic evaluation