FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning

2026-08-10

AI Summary

This work addresses the slow convergence of decentralized federated learning (DFL) under data heterogeneity, where a uniform learning rate struggles to balance global model consensus with local adaptation. To overcome this limitation, the authors propose FedA2L, the first method to enable layer-wise adaptive learning rates in DFL without incurring additional communication overhead. FedA2L dynamically adjusts per-layer learning rates by modeling inter-layer divergence, local update intensity, and network consensus constraints. This approach effectively mitigates optimization conflicts between foundational and task-specific layers. Empirical results demonstrate that FedA2L accelerates convergence by up to 4.94× compared to standard DFL and reduces communication rounds by up to 59% relative to scheduler-based baselines—all without extra communication—and maintains robust performance across large-scale, sparsely connected, and highly heterogeneous settings.

Abstract

Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL algorithms, various model architectures, and datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L.

Problem

• decentralized federated learning
• learning rate
• data heterogeneity
• layer-wise optimization
• convergence inefficiency

Innovation

• layer-wise learning rate
• decentralized federated learning
• model divergence
• adaptive optimization
• non-IID data