Protection against Source Inference Attacks in Federated Learning

2026-03-02

AI Summary

This work addresses the privacy threat posed by source inference attacks (SIAs) in federated learning, where an honest-but-curious central server can infer client identities from model updates. The study demonstrates for the first time that conventional shuffling mechanisms are insufficient to defend against such attacks. To mitigate this vulnerability, the authors propose a novel defense framework that integrates parameter-level shuffling with the Residue Number System (RNS). Within the shuffled model paradigm, this approach effectively disrupts the server’s ability to trace updates back to their originating clients, reducing SIA accuracy to the level of random guessing. Extensive evaluations across multiple models and datasets confirm that the method preserves high model utility while providing strong privacy guarantees, successfully balancing performance and confidentiality in federated settings.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated Learning (FL) was initially proposed as a privacy-preserving machine learning paradigm. However, FL has been shown to be susceptible to a series of privacy attacks. Recently, there has been concern about the Source Inference Attack (SIA), where an honest-but-curious central server attempts to identify exactly which client owns a given data point which was used in the training phase. Alarmingly, standard gradient obfuscation techniques with Differential Privacy have been shown to be ineffective against SIAs, at least without severely diminishing the accuracy. In this work, we propose a defense against SIAs within the widely studied shuffle model of FL, where an honest shuffler acts as an intermediary between the clients and the server. First, we demonstrate that standard naive shuffling alone is insufficient to prevent SIAs. To effectively defend against SIAs, shuffling needs to be applied at a more granular level; we propose a novel combination of parameter-level shuffling with the residue number system (RNS). Our approach provides robust protection against SIAs without affecting the accuracy of the joint model and can be seamlessly integrated into other privacy protection mechanisms. We conduct experiments on a series of models and datasets, confirming that standard shuffling approaches fail to prevent SIAs and that, in contrast, our proposed method reduce the attack's accuracy to the level of random guessing.

Problem

• Federated Learning
• Source Inference Attack
• Privacy
• Differential Privacy
• Shuffle Model

Innovation

• Source Inference Attack
• Parameter-level Shuffling
• Residue Number System
• Federated Learning
• Shuffle Model