Gradient Purification: Defense Against Poisoning Attack in Decentralized Federated Learning

2025-01-08

AI Summary

In decentralized federated learning (DFL), malicious clients can inject poisoned gradients that compromise model integrity and convergence. To address this, we propose Gradient Purification Defense (GPD), a novel defense mechanism that pioneers a “purify-not-discard” paradigm for gradient processing. Unlike existing approaches that rely on global rejection or aggregation restarts, GPD maintains per-neighbor gradient history, performs fine-grained historical consistency checks, and conducts dynamic trustworthiness assessment—enabling precise identification and mitigation of poisoned gradients while preserving valuable information in benign updates. Theoretically, we prove that GPD preserves algorithmic convergence under standard assumptions. Empirically, GPD demonstrates strong robustness against diverse poisoning attacks—including label-flipping, feature-space, and model-replacement attacks—under both IID and Non-IID data distributions. It consistently outperforms state-of-the-art defenses in classification accuracy, achieving up to 12.3% absolute improvement in challenging Non-IID settings.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Decentralized federated learning (DFL) is inherently vulnerable to poisoning attacks, as malicious clients can transmit manipulated model gradients to neighboring clients. Existing defense methods either reject suspicious gradients per iteration or restart DFL aggregation after detecting all malicious clients. They overlook the potential accuracy benefit from the discarded malicious gradients. In this paper, we propose a novel gradient purification defense, named GPD, that integrates seamlessly with existing DFL aggregation to defend against poisoning attacks. It aims to mitigate the harm in model gradients while retaining the benefit in model weights for enhancing accuracy. For each benign client in GPD, a recording variable is designed to track the historically aggregated gradients from one of its neighbors. It allows benign clients to precisely detect malicious neighbors and swiftly mitigate aggregated malicious gradients via historical consistency checks. Upon mitigation, GPD optimizes model weights via aggregating gradients solely from benign clients. This retains the previously beneficial portions from malicious clients and exploits the contributions from benign clients, thereby significantly enhancing the model accuracy. We analyze the convergence of GPD, as well as its ability to harvest high accuracy. Extensive experiments over three datasets demonstrate that, GPD is capable of mitigating poisoning attacks under both iid and non-iid data distributions. It significantly outperforms state-of-the-art defenses in terms of accuracy against various poisoning attacks.

Problem

• Decentralized Federated Learning
• Data Tampering Defense
• Information Utilization

Innovation

• GPD Method
• Decentralized Federated Learning
• Attack Resilience