AFBS:Buffer Gradient Selection in Semi-asynchronous Federated Learning

2025-06-15

AI Summary

To address performance degradation caused by gradient staleness in asynchronous federated learning, this paper proposes a privacy-preserving gradient selection mechanism operating within client-side buffers. The method operates under a semi-asynchronous framework and introduces three key components: (1) dynamic gradient selection based on information-theoretic value scoring, (2) encrypted client clustering via random projection of label distributions, and (3) integration of label distribution modeling, differential privacy–guaranteed clustering, and value-aware gradient selection to jointly optimize convergence and privacy. Experiments on CIFAR-100 demonstrate that the approach achieves a 4.8% improvement in test accuracy and reduces time-to-target accuracy by 75% compared to state-of-the-art baselines. These results significantly enhance both the efficiency and robustness of semi-asynchronous federated learning while preserving client-level privacy.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Asynchronous federated learning (AFL) accelerates training by eliminating the need to wait for stragglers, but its asynchronous nature introduces gradient staleness, where outdated gradients degrade performance. Existing solutions address this issue with gradient buffers, forming a semi-asynchronous framework. However, this approach struggles when buffers accumulate numerous stale gradients, as blindly aggregating all gradients can harm training. To address this, we propose AFBS (Asynchronous FL Buffer Selection), the first algorithm to perform gradient selection within buffers while ensuring privacy protection. Specifically, the client sends the random projection encrypted label distribution matrix before training, and the server performs client clustering based on it. During training, server scores and selects gradients within each cluster based on their informational value, discarding low-value gradients to enhance semi-asynchronous federated learning. Extensive experiments in highly heterogeneous system and data environments demonstrate AFBS's superior performance compared to state-of-the-art methods. Notably, on the most challenging task, CIFAR-100, AFBS improves accuracy by up to 4.8% over the previous best algorithm and reduces the time to reach target accuracy by 75%.

Problem

• Address gradient staleness in asynchronous federated learning
• Select high-value gradients to improve training efficiency
• Enhance privacy-protected semi-asynchronous federated learning performance

Innovation

• Client clustering via encrypted label distribution
• Gradient selection based on informational value
• Privacy-preserving semi-asynchronous federated learning