Client Selection in Federated Learning with Data Heterogeneity and Network Latencies

2025-04-02

AI Summary

In federated learning, the coexistence of statistical heterogeneity (non-IID data) and system heterogeneity (network latency) severely impedes convergence speed. Existing approaches address only one type of heterogeneity in isolation, lacking a unified optimization framework. This paper presents the first joint modeling of both heterogeneities and proposes two theoretically optimal client selection strategies per round. By minimizing the theoretical convergence time, we derive lightweight, efficiently solvable optimization problems that integrate latency-aware sampling, non-IID-adaptive training, and unified convergence analysis for both convex and non-convex objectives. Extensive experiments across nine non-IID datasets, two realistic latency distributions, and non-convex neural networks demonstrate that our method accelerates convergence by up to 20× over state-of-the-art baselines while ensuring stable convergence.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated learning (FL) is a distributed machine learning paradigm where multiple clients conduct local training based on their private data, then the updated models are sent to a central server for global aggregation. The practical convergence of FL is challenged by multiple factors, with the primary hurdle being the heterogeneity among clients. This heterogeneity manifests as data heterogeneity concerning local data distribution and latency heterogeneity during model transmission to the server. While prior research has introduced various efficient client selection methods to alleviate the negative impacts of either of these heterogeneities individually, efficient methods to handle real-world settings where both these heterogeneities exist simultaneously do not exist. In this paper, we propose two novel theoretically optimal client selection schemes that can handle both these heterogeneities. Our methods involve solving simple optimization problems every round obtained by minimizing the theoretical runtime to convergence. Empirical evaluations on 9 datasets with non-iid data distributions, 2 practical delay distributions, and non-convex neural network models demonstrate that our algorithms are at least competitive to and at most 20 times better than best existing baselines.

Problem

• Handling data and latency heterogeneity in federated learning
• Developing optimal client selection for heterogeneous FL settings
• Improving convergence speed in non-iid and delayed FL scenarios

Innovation

• Optimal client selection for data and latency heterogeneity
• Minimizes theoretical runtime to convergence
• Outperforms baselines by up to 20 times