Extending Information Bottleneck Attribution to Video Sequences

2025-01-28

AI Summary

To address the lack of interpretability in deepfake video detection models, this paper proposes Video Information Bottleneck Attribution (VIBA), the first extension of Information Bottleneck Attribution (IBA) to the spatiotemporal domain. VIBA introduces a gradient-free dual-path attribution framework: it jointly leverages Xception for spatial feature extraction and VGG11 augmented with optical flow modeling to capture temporal dynamics, thereby generating spatial-temporal saliency maps and optical flow attribution maps in a temporally consistent manner. Evaluated on a custom deepfake video dataset, VIBA achieves a 23.7% improvement in Intersection-over-Union (IoU) for localizing forged regions compared to image-level attribution methods, demonstrating strong agreement with human annotations. This work establishes the first IBA-based paradigm tailored for temporal modeling in video-level explainable AI, significantly enhancing the trustworthiness and debuggability of dynamic forgery detection.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce VIBA, a novel approach for explainable video classification by adapting Information Bottlenecks for Attribution (IBA) to video sequences. While most traditional explainability methods are designed for image models, our IBA framework addresses the need for explainability in temporal models used for video analysis. To demonstrate its effectiveness, we apply VIBA to video deepfake detection, testing it on two architectures: the Xception model for spatial features and a VGG11-based model for capturing motion dynamics through optical flow. Using a custom dataset that reflects recent deepfake generation techniques, we adapt IBA to create relevance and optical flow maps, visually highlighting manipulated regions and motion inconsistencies. Our results show that VIBA generates temporally and spatially consistent explanations, which align closely with human annotations, thus providing interpretability for video classification and particularly for deepfake detection.

Problem

• Video Classification
• Deep Fakes
• Decision Interpretation

Innovation

• Information Bottleneck
• Video Understanding
• Deepfake Detection