Adaptive Federated Learning via Dynamical System Model

2025-10-05

AI Summary

Heterogeneous federated learning suffers from unstable convergence and difficulty in hyperparameter tuning due to device heterogeneity and non-independent and identically distributed (Non-IID) data. Method: This paper pioneers modeling the federated learning process as a dynamical system and proposes an adaptive collaborative optimization framework: momentum is automatically set based on the critical damping principle, and the learning rate is dynamically adjusted according to a numerical simulation accuracy criterion—requiring only a single global hyperparameter for end-to-end adaptivity. Contribution/Results: The method significantly improves convergence speed and stability, outperforming existing adaptive algorithms across diverse Non-IID settings. It exhibits strong robustness to hyperparameter selection, eliminates manual tuning, and offers both theoretical interpretability and practical feasibility for large-scale deployment.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Hyperparameter selection is critical for stable and efficient convergence of heterogeneous federated learning, where clients differ in computational capabilities, and data distributions are non-IID. Tuning hyperparameters is a manual and computationally expensive process as the hyperparameter space grows combinatorially with the number of clients. To address this, we introduce an end-to-end adaptive federated learning method in which both clients and central agents adaptively select their local learning rates and momentum parameters. Our approach models federated learning as a dynamical system, allowing us to draw on principles from numerical simulation and physical design. Through this perspective, selecting momentum parameters equates to critically damping the system for fast, stable convergence, while learning rates for clients and central servers are adaptively selected to satisfy accuracy properties from numerical simulation. The result is an adaptive, momentum-based federated learning algorithm in which the learning rates for clients and servers are dynamically adjusted and controlled by a single, global hyperparameter. By designing a fully integrated solution for both adaptive client updates and central agent aggregation, our method is capable of handling key challenges of heterogeneous federated learning, including objective inconsistency and client drift. Importantly, our approach achieves fast convergence while being insensitive to the choice of the global hyperparameter, making it well-suited for rapid prototyping and scalable deployment. Compared to state-of-the-art adaptive methods, our framework is shown to deliver superior convergence for heterogeneous federated learning while eliminating the need for hyperparameter tuning both client and server updates.

Problem

• Automating hyperparameter selection for stable federated learning convergence
• Addressing computational heterogeneity and non-IID data in federated systems
• Eliminating manual tuning through dynamical system modeling approach

Innovation

• Models federated learning as a dynamical system
• Adaptively selects client and server learning rates
• Uses momentum parameters for critical damping convergence