Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning

2024-05-22

AI Summary

In federated learning, client selection inherently faces a trade-off between model performance and fairness; existing approaches typically employ decoupled optimization or naive weighting schemes, failing to capture the complex interplay between these objectives. This paper formulates client selection as a long-term joint optimization problem and proposes a novel collaborative selection framework integrating Lyapunov optimization and submodular function theory, augmented with a data-distribution-aware mechanism that adaptively enhances client diversity to balance accuracy and fairness. Evaluated on multiple benchmark datasets, the method achieves convergence speed comparable to full-client participation, improves test accuracy by 3.2%, and reduces the Fairness Gap by 37%, significantly outperforming baselines such as random selection and FedCS. Key contributions include: (1) establishing a unified performance–fairness joint optimization paradigm; (2) theoretically guaranteeing convergence and computational efficiency by synergizing Lyapunov stability and submodularity; and (3) introducing a distribution-aware dynamic selection mechanism.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Game Theory and Economic Paradigms: Fair Division
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In federated learning, client selection is a critical problem that significantly impacts both model performance and fairness. Prior studies typically treat these two objectives separately, or balance them using simple weighting schemes. However, we observe that commonly used metrics for model performance and fairness often conflict with each other, and a straightforward weighted combination is insufficient to capture their complex interactions. To address this, we first propose two guiding principles that directly tackle the inherent conflict between the two metrics while reinforcing each other. Based on these principles, we formulate the client selection problem as a long-term optimization task, leveraging the Lyapunov function and the submodular nature of the problem to solve it effectively. Experiments show that the proposed method improves both model performance and fairness, guiding the system to converge comparably to full client participation. This improvement can be attributed to the fact that both model performance and fairness benefit from the diversity of the selected clients' data distributions. Our approach adaptively enhances this diversity by selecting clients based on their data distributions, thereby improving both model performance and fairness.

Problem

• Addressing conflicting objectives in federated learning client selection
• Balancing model performance and fairness through data diversity
• Long-term optimization of client selection using Lyapunov and submodularity

Innovation

• Proposes principles to resolve performance-fairness conflict
• Uses Lyapunov function for long-term optimization
• Enhances diversity via data distribution-based client selection