Enhancing Security and Privacy in Federated Learning using Low-Dimensional Update Representation and Proximity-Based Defense

2024-05-29

IEEE Transactions on Knowledge and Data Engineering

AI Summary

Federated learning (FL) faces dual threats of privacy leakage and Byzantine attacks under mutual distrust between clients and servers. This paper proposes FLURP, the first framework integrating low-dimensional update representation (LUR) with a lightweight proximity-based defense leveraging shared distance matrices, achieving end-to-end privacy via optimized secure multi-party computation (SMPC) while efficiently identifying malicious model updates. Key innovations include an ℓ∞-norm sliding-window compression scheme and joint distance-matrix computation, reducing SMPC communication overhead by three orders of magnitude. Extensive experiments demonstrate that FLURP maintains high accuracy and strong robustness against diverse Byzantine attacks, significantly outperforms state-of-the-art methods in both communication and computational efficiency, and exhibits excellent scalability.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated Learning (FL) is a promising privacy-preserving machine learning paradigm that allows data owners to collaboratively train models while keeping their data localized. Despite its potential, FL faces challenges related to the trustworthiness of both clients and servers, particularly against curious or malicious adversaries. In this paper, we introduce a novel framework named underline{F}ederated underline{L}earning with Low-Dimensional underline{U}pdate underline{R}epresentation and underline{P}roximity-Based defense (FLURP), designed to address privacy preservation and resistance to Byzantine attacks in distributed learning environments. FLURP employs $mathsf{LinfSample}$ method, enabling clients to compute the $l_{infty}$ norm across sliding windows of updates, resulting in a Low-Dimensional Update Representation (LUR). Calculating the shared distance matrix among LURs, rather than updates, significantly reduces the overhead of Secure Multi-Party Computation (SMPC) by three orders of magnitude while effectively distinguishing between benign and poisoned updates. Additionally, FLURP integrates a privacy-preserving proximity-based defense mechanism utilizing optimized SMPC protocols to minimize communication rounds. Our experiments demonstrate FLURP's effectiveness in countering Byzantine adversaries with low communication and runtime overhead. FLURP offers a scalable framework for secure and reliable FL in distributed environments, facilitating its application in scenarios requiring robust data management and security.

Problem

• Enhance privacy in Federated Learning
• Defend against Byzantine attacks
• Reduce Secure Multi-Party Computation overhead

Innovation

• Low-Dimensional Update Representation
• Proximity-Based Defense Mechanism
• Secure Multi-Party Computation