extit{FedABC}: Attention-Based Client Selection for Federated Learning with Long-Term View

2025-07-28

AI Summary

To address slow convergence, low accuracy, and excessive communication/computation overhead in federated learning for 6G-native AI—caused by client data heterogeneity and frequent participation—this paper proposes a long-horizon attention-driven client selection algorithm. The method innovatively employs an attention mechanism to quantify model similarity and individual client contribution, jointly modeling global model evolution dynamics; it adaptively adjusts the selection threshold based on a “late-stage prioritization” principle. This enables tight coordination between client selection and global optimization. Experiments on CIFAR-10 demonstrate that, compared to FedAvg, the proposed method reduces client participation by 32% while maintaining comparable accuracy; against state-of-the-art approaches, it improves accuracy by 3.5% and further reduces participating clients by 2%.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Humans and AI: Human-in-the-loop Machine Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Native AI support is a key objective in the evolution of 6G networks, with Federated Learning (FL) emerging as a promising paradigm. FL allows decentralized clients to collaboratively train an AI model without directly sharing their data, preserving privacy. Clients train local models on private data and share model updates, which a central server aggregates to refine the global model and redistribute it for the next iteration. However, client data heterogeneity slows convergence and reduces model accuracy, and frequent client participation imposes communication and computational burdens. To address these challenges, we propose extit{FedABC}, an innovative client selection algorithm designed to take a long-term view in managing data heterogeneity and optimizing client participation. Inspired by attention mechanisms, extit{FedABC} prioritizes informative clients by evaluating both model similarity and each model's unique contributions to the global model. Moreover, considering the evolving demands of the global model, we formulate an optimization problem to guide extit{FedABC} throughout the training process. Following the ``later-is-better" principle, extit{FedABC} adaptively adjusts the client selection threshold, encouraging greater participation in later training stages. Extensive simulations on CIFAR-10 demonstrate that extit{FedABC} significantly outperforms existing approaches in model accuracy and client participation efficiency, achieving comparable performance with 32% fewer clients than the classical FL algorithm extit{FedAvg}, and 3.5% higher accuracy with 2% fewer clients than the state-of-the-art. This work marks a step toward deploying FL in heterogeneous, resource-constrained environments, thereby supporting native AI capabilities in 6G networks.

Problem

• Address data heterogeneity in Federated Learning to improve model accuracy
• Optimize client participation to reduce communication and computational burdens
• Enhance long-term client selection using attention mechanisms for efficiency

Innovation

• Attention-based client selection for FL
• Long-term data heterogeneity management
• Adaptive client participation optimization