🤖 AI Summary
To address three key challenges in federated learning—statistical heterogeneity (non-IID data), computational asynchrony across clients, and constrained communication bandwidth—this paper proposes the first FedAvg variant supporting *simultaneous* data heterogeneity, partial-client asynchronous updates, and gradient compression. Methodologically, it introduces a unified framework modeling all three sources of heterogeneity, integrating local asynchronous scheduling, error-compensated quantized compression, and adaptive aggregation. Theoretically, we provide a rigorous convergence analysis proving that the algorithm achieves the same asymptotic convergence rate as FedAvg under broad parameter conditions. Empirically, extensive experiments on the LEAF benchmark (up to 300 clients) demonstrate that our method significantly outperforms existing quantized and asynchronous baselines in both convergence speed and communication efficiency.
📝 Abstract
Federated Learning (FL) enables large-scale distributed training of machine learning models, while still allowing individual nodes to maintain data locally. However, executing FL at scale comes with inherent practical challenges: 1) heterogeneity of the local node data distributions, 2) heterogeneity of node computational speeds (asynchrony), but also 3) constraints in the amount of communication between the clients and the server. In this work, we present the first variant of the classic federated averaging (FedAvg) algorithm which, at the same time, supports data heterogeneity, partial client asynchrony, and communication compression. Our algorithm comes with a novel, rigorous analysis showing that, in spite of these system relaxations, it can provide similar convergence to FedAvg in interesting parameter regimes. Experimental results in the rigorous LEAF benchmark on setups of up to 300 nodes show that our algorithm ensures fast convergence for standard federated tasks, improving upon prior quantized and asynchronous approaches.
