From Research to Reality: Feasibility of Gradient Inversion Attacks in Federated Learning

2025-08-27

AI Summary

Gradient inversion attacks in federated learning have long been overlooked during the training phase, with existing studies predominantly focusing on inference-time scenarios. Method: This paper presents the first systematic empirical evaluation of gradient inversion feasibility during training, analyzing how architectural choices (e.g., skip connections, pre-activation normalization) and training behaviors (e.g., Dropout, BatchNorm) critically affect attack success. We propose two novel gradient inversion attacks specifically designed for the training phase and demonstrate their first successful application on production-grade object detection models (e.g., YOLOv5). Results: Experiments reveal that effective inversion occurs only when models simultaneously exhibit shallow depth, wide width, skip connections, and pre-activation normalization—characteristics absent in mainstream production models, which thus inherently possess strong robustness. Our work establishes the first empirical benchmark for privacy risk assessment during federated training and provides actionable architectural security guidelines for privacy-preserving model design.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Gradient inversion attacks have garnered attention for their ability to compromise privacy in federated learning. However, many studies consider attacks with the model in inference mode, where training-time behaviors like dropout are disabled and batch normalization relies on fixed statistics. In this work, we systematically analyze how architecture and training behavior affect vulnerability, including the first in-depth study of inference-mode clients, which we show dramatically simplifies inversion. To assess attack feasibility under more realistic conditions, we turn to clients operating in standard training mode. In this setting, we find that successful attacks are only possible when several architectural conditions are met simultaneously: models must be shallow and wide, use skip connections, and, critically, employ pre-activation normalization. We introduce two novel attacks against models in training-mode with varying attacker knowledge, achieving state-of-the-art performance under realistic training conditions. We extend these efforts by presenting the first attack on a production-grade object-detection model. Here, to enable any visibly identifiable leakage, we revert to the lenient inference mode setting and make multiple architectural modifications to increase model vulnerability, with the extent of required changes highlighting the strong inherent robustness of such architectures. We conclude this work by offering the first comprehensive mapping of settings, clarifying which combinations of architectural choices and operational modes meaningfully impact privacy. Our analysis provides actionable insight into when models are likely vulnerable, when they appear robust, and where subtle leakage may persist. Together, these findings reframe how gradient inversion risk should be assessed in future research and deployment scenarios.

Problem

• Assessing gradient inversion attack feasibility in realistic federated learning settings
• Analyzing how architecture and training behavior affect model vulnerability
• Providing comprehensive mapping of architectural choices impacting privacy risks

Innovation

• Novel attacks for training-mode clients with varying knowledge
• First attack on production-grade object detection model
• Comprehensive mapping of architectural and operational vulnerabilities