VGMShield: Mitigating Misuse of Video Generative Models

2024-02-20

arXiv.org

AI Summary

To address the risks of disinformation and malicious dissemination arising from the abuse of video generation models, this paper proposes the first end-to-end defense framework encompassing detection, provenance attribution, and prevention. Methodologically: (1) We leverage pre-trained spatiotemporal models (e.g., VideoMAE, TimeSformer) to extract dynamic anomaly features and employ lightweight classifiers for high-accuracy detection and model attribution; (2) We identify intrinsic architectural deficiencies in video generative models’ spatiotemporal modeling and design a gradient-guided imperceptible perturbation mechanism to actively degrade video authenticity. Experiments across seven mainstream open-source video generation models achieve near-perfect detection and attribution accuracy (>99.5%). Moreover, the prevention module significantly reduces human-perceived realism of generated videos—quantified via perceptual evaluation—thereby mitigating their potential for malicious propagation. This work establishes the first holistic, three-stage defense paradigm against synthetic video misuse.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Responsible Web: Mitigating misinformation and disinformation

Abstract

With the rapid advancement in video generation, people can conveniently utilize video generation models to create videos tailored to their specific desires. Nevertheless, there are also growing concerns about their potential misuse in creating and disseminating false information. In this work, we introduce VGMShield: a set of three straightforward but pioneering mitigations through the lifecycle of fake video generation. We start from extit{fake video detection} trying to understand whether there is uniqueness in generated videos and whether we can differentiate them from real videos; then, we investigate the extit{tracing} problem, which maps a fake video back to a model that generates it. Towards these, we propose to leverage pre-trained models that focus on {it spatial-temporal dynamics} as the backbone to identify inconsistencies in videos. Through experiments on seven state-of-the-art open-source models, we demonstrate that current models still cannot perfectly handle spatial-temporal relationships, and thus, we can accomplish detection and tracing with nearly perfect accuracy. Furthermore, anticipating future generative model improvements, we propose a {it prevention} method that adds invisible perturbations to images to make the generated videos look unreal. Together with fake video detection and tracing, our multi-faceted set of solutions can effectively mitigate misuse of video generative models.

Problem

• Detecting fake videos by identifying spatial-temporal inconsistencies
• Tracing fake video sources to the generating model
• Preventing misuse via invisible perturbations in query images

Innovation

• Leverages spatial-temporal dynamics for video detection
• Identifies patterns in attention shifts and motion
• Uses invisible perturbations to prevent fake videos