Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

2025-06-11

AI Summary

In heterogeneous federated learning, achieving simultaneous Byzantine robustness and client privacy remains challenging due to inherent incompatibilities between preprocessing-based robust aggregation and cryptographic privacy mechanisms. Method: This paper proposes a novel multi-stage collaborative framework that— for the first time—integrates verifiable secret sharing (VSS), secure aggregation (SecAgg), and symmetric private information retrieval (SPIR) to jointly optimize information-theoretic privacy guarantees and defense against malicious clients. The design eliminates the compatibility bottleneck of conventional preprocessing approaches and enables zeroth-order gradient estimation to reduce communication overhead. Contribution/Results: Extensive experiments demonstrate that the proposed scheme significantly outperforms state-of-the-art methods under diverse Byzantine attacks—including omniscient, label-flipping, and model-poisoning adversaries—while preserving information-theoretic privacy, improving communication efficiency, and enhancing model convergence stability.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Ensuring resilience to Byzantine clients while maintaining the privacy of the clients' data is a fundamental challenge in federated learning (FL). When the clients' data is homogeneous, suitable countermeasures were studied from an information-theoretic perspective utilizing secure aggregation techniques while ensuring robust aggregation of the clients' gradients. However, the countermeasures used fail when the clients' data is heterogeneous. Suitable pre-processing techniques, such as nearest neighbor mixing, were recently shown to enhance the performance of those countermeasures in the heterogeneous setting. Nevertheless, those pre-processing techniques cannot be applied with the introduced privacy-preserving mechanisms. We propose a multi-stage method encompassing a careful co-design of verifiable secret sharing, secure aggregation, and a tailored symmetric private information retrieval scheme to achieve information-theoretic privacy guarantees and Byzantine resilience under data heterogeneity. We evaluate the effectiveness of our scheme on a variety of attacks and show how it outperforms the previously known techniques. Since the communication overhead of secure aggregation is non-negligible, we investigate the interplay with zero-order estimation methods that reduce the communication cost in state-of-the-art FL tasks and thereby make private aggregation scalable.

Problem

• Ensuring Byzantine resilience and privacy in heterogeneous federated learning
• Overcoming limitations of existing countermeasures for heterogeneous client data
• Reducing communication overhead in private aggregation for scalable FL

Innovation

• Multi-stage verifiable secret sharing
• Secure aggregation with privacy guarantees
• Zero-order estimation for scalability