Timely Parameter Updating in Over-the-Air Federated Learning

2025-12-22

AI Summary

In over-the-air computation-based federated learning (OAC-FL), limited orthogonal waveform resources conflict with the impracticality of transmitting high-dimensional gradients in full. Method: This paper proposes FAIR-k, the first algorithm jointly leveraging gradient magnitude and update freshness to dynamically select a top-k subset for analog aggregation over the air; it models parameter staleness via a Markov chain and theoretically analyzes convergence under non-uniform Lipschitz continuity. Contribution/Results: We establish the joint impact of data heterogeneity, channel noise, and staleness on convergence. Theoretically, FAIR-k significantly accelerates convergence and improves communication efficiency, enables longer local training epochs without accuracy loss, and maintains robust performance under heterogeneous data distributions and noisy wireless channels.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Incorporating over-the-air computations (OAC) into the model training process of federated learning (FL) is an effective approach to alleviating the communication bottleneck in FL systems. Under OAC-FL, every client modulates its intermediate parameters, such as gradient, onto the same set of orthogonal waveforms and simultaneously transmits the radio signal to the edge server. By exploiting the superposition property of multiple-access channels, the edge server can obtain an automatically aggregated global gradient from the received signal. However, the limited number of orthogonal waveforms available in practical systems is fundamentally mismatched with the high dimensionality of modern deep learning models. To address this issue, we propose Freshness Freshness-mAgnItude awaRe top-k (FAIR-k), an algorithm that selects, in each communication round, the most impactful subset of gradients to be updated over the air. In essence, FAIR-k combines the complementary strengths of the Round-Robin and Top-k algorithms, striking a delicate balance between timeliness (freshness of parameter updates) and importance (gradient magnitude). Leveraging tools from Markov analysis, we characterize the distribution of parameter staleness under FAIR-k. Building on this, we establish the convergence rate of OAC-FL with FAIR-k, which discloses the joint effect of data heterogeneity, channel noise, and parameter staleness on the training efficiency. Notably, as opposed to conventional analyses that assume a universal Lipschitz constant across all the clients, our framework adopts a finer-grained model of the data heterogeneity. The analysis demonstrates that since FAIR-k promotes fresh (and fair) parameter updates, it not only accelerates convergence but also enhances communication efficiency by enabling an extended period of local training without significantly affecting overall training efficiency.

Problem

• Addresses the mismatch between limited orthogonal waveforms and high-dimensional models in over-the-air federated learning.
• Proposes FAIR-k algorithm to balance timeliness and importance of gradient updates for efficient training.
• Analyzes convergence rates considering data heterogeneity, channel noise, and parameter staleness effects.

Innovation

• FAIR-k algorithm selects impactful gradients for over-the-air updates
• Combines Round-Robin and Top-k methods for timeliness and importance
• Uses Markov analysis to model parameter staleness and convergence rates