Divergence-Based Adaptive Aggregation for Byzantine Robust Federated Learning

2026-01-11

arXiv.org

AI Summary

This work addresses the dual challenges of client drift caused by data heterogeneity and system vulnerability under Byzantine attacks in federated learning. The authors propose DRAG and its robust variant BR-DRAG, which introduce a reference direction and a deviation metric to linearly calibrate local updates, thereby mitigating drift. In Byzantine settings, the server leverages a trusted root dataset to construct a robust reference direction for efficient aggregation. Notably, this is the first approach to integrate deviation-aware metrics with an adaptive calibration mechanism, simultaneously tackling data heterogeneity and malicious attacks without increasing communication overhead. The framework supports non-convex models and partial client participation while ensuring fast convergence. Experimental results demonstrate that DRAG effectively alleviates client drift, and BR-DRAG consistently maintains high robustness and convergence speed across diverse attack scenarios and heterogeneous data distributions.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Inherent client drifts caused by data heterogeneity, as well as vulnerability to Byzantine attacks within the system, hinder effective model training and convergence in federated learning (FL). This paper presents two new frameworks, named DiveRgence-based Adaptive aGgregation (DRAG) and Byzantine-Resilient DRAG (BR-DRAG), to mitigate client drifts and resist attacks while expediting training. DRAG designs a reference direction and a metric named divergence of degree to quantify the deviation of local updates. Accordingly, each worker can align its local update via linear calibration without extra communication cost. BR-DRAG refines DRAG under Byzantine attacks by maintaining a vetted root dataset at the server to produce trusted reference directions. The workers'updates can be then calibrated to mitigate divergence caused by malicious attacks. We analytically prove that DRAG and BR-DRAG achieve fast convergence for non-convex models under partial worker participation, data heterogeneity, and Byzantine attacks. Experiments validate the effectiveness of DRAG and its superior performance over state-of-the-art methods in handling client drifts, and highlight the robustness of BR-DRAG in maintaining resilience against data heterogeneity and diverse Byzantine attacks.

Problem

• client drift
• data heterogeneity
• Byzantine attacks
• federated learning

Innovation

• Byzantine robustness
• federated learning
• client drift
• adaptive aggregation
• divergence metric