Images in Motion?: A First Look into Video Leakage in Collaborative Deep Learning

2025-09-11

AI Summary

This paper presents the first systematic study of gradient inversion attacks targeting video data in federated learning (FL), addressing a critical gap: prior work focuses on images, text, and tabular data, while video-specific privacy risks remain unexplored. To assess reconstructability under gradient leakage, we propose a multi-frame reference comparative analysis framework that integrates pretrained feature extractors with super-resolution reconstruction techniques. Our contributions are threefold: (1) We demonstrate that original video frames can be reconstructed with high fidelity—even from shared gradients alone—establishing video gradient leakage as a novel, practical privacy threat in FL. (2) While pretrained feature extractors significantly improve model robustness against inversion, end-to-end lightweight classifiers still leak sensitive temporal information. (3) Super-resolution enhancement substantially amplifies reconstruction quality, exposing structural vulnerabilities inherent to FL-based video training pipelines.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Privacy
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Large-scale security measurements

Abstract

Federated learning (FL) allows multiple entities to train a shared model collaboratively. Its core, privacy-preserving principle is that participants only exchange model updates, such as gradients, and never their raw, sensitive data. This approach is fundamental for applications in domains where privacy and confidentiality are important. However, the security of this very mechanism is threatened by gradient inversion attacks, which can reverse-engineer private training data directly from the shared gradients, defeating the purpose of FL. While the impact of these attacks is known for image, text, and tabular data, their effect on video data remains an unexamined area of research. This paper presents the first analysis of video data leakage in FL using gradient inversion attacks. We evaluate two common video classification approaches: one employing pre-trained feature extractors and another that processes raw video frames with simple transformations. Our initial results indicate that the use of feature extractors offers greater resilience against gradient inversion attacks. We also demonstrate that image super-resolution techniques can enhance the frames extracted through gradient inversion attacks, enabling attackers to reconstruct higher-quality videos. Our experiments validate this across scenarios where the attacker has access to zero, one, or more reference frames from the target environment. We find that although feature extractors make attacks more challenging, leakage is still possible if the classifier lacks sufficient complexity. We, therefore, conclude that video data leakage in FL is a viable threat, and the conditions under which it occurs warrant further investigation.

Problem

• Investigating video data leakage via gradient inversion attacks in federated learning
• Evaluating resilience of video classification approaches against privacy breaches
• Assessing conditions enabling video reconstruction from shared gradients

Innovation

• Evaluates video leakage via gradient inversion attacks
• Uses feature extractors and raw frame processing
• Applies super-resolution to enhance reconstructed videos