FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization

2024-05-10

International Joint Conference on Artificial Intelligence

AI Summary

Federated learning faces significant challenges including high statistical and system heterogeneity, as well as substantial communication and energy overheads; existing client selection methods struggle to jointly optimize accuracy, latency, and energy efficiency. This paper pioneers a generative formulation of client selection and proposes a differentiable framework grounded in a continuous representation space: an encoder–evaluator–decoder architecture enables gradient-based optimization of continuous representations, followed by beam search to generate high-quality client subsets. Innovatively integrating large-model principles, we construct a generalizable and differentiable representation space that supports multi-objective co-optimization. Extensive experiments across multiple benchmarks demonstrate that our method outperforms both heuristic and learning-based baselines in model accuracy, reduces communication rounds by 18%, lowers edge-side energy consumption by 23%, and exhibits strong cross-scenario generalization capability.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated Learning faces significant challenges in statistical and system heterogeneity, along with high energy consumption, necessitating efficient client selection strategies. Traditional approaches, including heuristic and learning-based methods, fall short of addressing these complexities holistically. In response, we propose FedGCS, a novel generative client selection framework that innovatively recasts the client selection process as a generative task. Drawing inspiration from the methodologies used in large language models, FedGCS efficiently encodes abundant decision-making knowledge within a continuous representation space, enabling efficient gradient-based optimization to search for optimal client selection that will be finally output via generation. The framework comprises four steps: (1) automatic collection of diverse “selection-score” pair data using classical client selection methods; (2) training an encoder-evaluator-decoder framework on this data to construct a continuous representation space; (3) employing gradient-based optimization in this space for optimal client selection; (4) generating the final optimal client selection via using beam search for the well-trained decoder. FedGCS outperforms traditional methods by being more comprehensive, generalizable, and efficient, simultaneously optimizing for model performance, latency, and energy consumption. The effectiveness of FedGCS is proven through extensive experimental analyses.

Problem

• Addressing statistical and system heterogeneity in federated learning
• Reducing high energy consumption through efficient client selection
• Optimizing model performance, latency, and energy consumption simultaneously

Innovation

• Framing client selection as generative task
• Encoding decisions in continuous representation space
• Optimizing selections via gradient-based methods