Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible

2025-04-05

AI Summary

In asynchronous federated learning, stale client model versions cause gradient conflicts and accuracy degradation. To address this, we propose FedADT—a knowledge distillation-based asynchronous version correction algorithm. Prior to gradient aggregation, FedADT dynamically corrects lagging local models using the most recent global model. It introduces the first version-aware correction mechanism tailored for asynchronous settings and designs an adaptive weighted knowledge distillation strategy to mitigate misleading guidance arising from suboptimal early-stage global models. Extensive experiments demonstrate that FedADT achieves the fastest convergence and highest final accuracy across multiple benchmark datasets, significantly outperforming state-of-the-art asynchronous methods (e.g., AsyncFedAvg, SCAFFOLD-A). Crucially, it incurs no additional computational overhead, ensuring both robustness and efficiency without compromising training speed or resource usage.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

As an emerging paradigm of federated learning, asynchronous federated learning offers significant speed advantages over traditional synchronous federated learning. Unlike synchronous federated learning, which requires waiting for all clients to complete updates before aggregation, asynchronous federated learning aggregates the models that have arrived in realtime, greatly improving training speed. However, this mechanism also introduces the issue of client model version inconsistency. When the differences between models of different versions during aggregation become too large, it may lead to conflicts, thereby reducing the models accuracy. To address this issue, this paper proposes an asynchronous federated learning version correction algorithm based on knowledge distillation, named FedADT. FedADT applies knowledge distillation before aggregating gradients, using the latest global model to correct outdated information, thus effectively reducing the negative impact of outdated gradients on the training process. Additionally, FedADT introduces an adaptive weighting function that adjusts the knowledge distillation weight according to different stages of training, helps mitigate the misleading effects caused by the poorer performance of the global model in the early stages of training. This method significantly improves the overall performance of asynchronous federated learning without adding excessive computational overhead. We conducted experimental comparisons with several classical algorithms, and the results demonstrate that FedADT achieves significant improvements over other asynchronous methods and outperforms all methods in terms of convergence speed.

Problem

• Addresses client model version inconsistency in asynchronous federated learning
• Reduces negative impact of outdated gradients using knowledge distillation
• Improves convergence speed and accuracy without excessive computational overhead

Innovation

• Knowledge distillation corrects outdated client models
• Adaptive weighting adjusts distillation during training
• FedADT enhances accuracy without extra computation