Client-Centric Federated Adaptive Optimization

2025-01-17

AI Summary

In federated learning, statistical and system heterogeneity across clients causes model drift and insufficient optimization adaptability. Method: This paper proposes the first client-centric adaptive optimization framework supporting full-stack realistic constraints—dynamic participation, asynchronous aggregation, and non-uniform local computation. By establishing the first systematic model of heterogeneity, it designs an adaptive update mechanism that simultaneously ensures theoretical convergence guarantees and engineering robustness. Contribution/Results: Under non-convex settings, the method achieves optimal convergence rates; its theoretical analysis rigorously quantifies how asynchrony and heterogeneity impact convergence. Extensive experiments on multiple standard benchmarks demonstrate significant improvements over state-of-the-art baselines, achieving dual gains in both convergence speed and generalization performance.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Non-convex Optimization
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private. With an increasing scale of clients and models, FL encounters two key challenges, client drift due to a high degree of statistical/system heterogeneity, and lack of adaptivity. However, most existing FL research is based on unrealistic assumptions that virtually ignore system heterogeneity. In this paper, we propose Client-Centric Federated Adaptive Optimization, which is a class of novel federated adaptive optimization approaches. We enable several features in this framework such as arbitrary client participation, asynchronous server aggregation, and heterogeneous local computing, which are ubiquitous in real-world FL systems but are missed in most existing works. We provide a rigorous convergence analysis of our proposed framework for general nonconvex objectives, which is shown to converge with the best-known rate. Extensive experiments show that our approaches consistently outperform the baseline by a large margin across benchmarks.

Problem

• Federated Learning
• Statistical Heterogeneity
• System Heterogeneity

Innovation

• Client-Centric Federated Learning
• Adaptive Optimization
• Heterogeneity Handling