FedEL: Federated Elastic Learning for Heterogeneous Devices

2025-09-20

AI Summary

In federated learning, hardware heterogeneity across devices causes stragglers to impede global training convergence; existing acceleration strategies—such as client selection, asynchronous updates, and local training—often compromise model accuracy or introduce gradient bias. To address this, we propose an elastic learning framework that enables piecewise model training via a sliding-window mechanism and incorporates a dynamic tensor importance selection module to adaptively prune non-critical parameters on resource-constrained devices, jointly optimizing local update quality and global convergence efficiency. Furthermore, the framework introduces runtime budget allocation and tensor importance alignment to mitigate biases arising from dual heterogeneity in data and device capabilities. Extensive experiments demonstrate that our approach achieves a 3.87× improvement in time-accuracy trade-off over state-of-the-art baselines while maintaining comparable test accuracy.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated learning (FL) enables distributed devices to collaboratively train machine learning models while maintaining data privacy. However, the heterogeneous hardware capabilities of devices often result in significant training delays, as straggler clients with limited resources prolong the aggregation process. Existing solutions such as client selection, asynchronous FL, and partial training partially address these challenges but encounter issues such as reduced accuracy, stale updates, and compromised model performance due to inconsistent training contributions. To overcome these limitations, we propose FedEL, a federated elastic learning framework that enhances training efficiency while maintaining model accuracy. FedEL introduces a novel window-based training process, sliding the window to locate the training part of the model and dynamically selecting important tensors for training within a coordinated runtime budget. This approach ensures progressive and balanced training across all clients, including stragglers. Additionally, FedEL employs a tensor importance adjustment module, harmonizing local and global tensor importance to mitigate biases caused by data heterogeneity. The experiment results show that FedEL achieves up to 3.87x improvement in time-to-accuracy compared to baselines while maintaining or exceeding final test accuracy.

Problem

• Addresses training delays from heterogeneous device capabilities in federated learning
• Overcomes accuracy loss and stale updates in existing FL solutions
• Ensures balanced training and mitigates data heterogeneity biases

Innovation

• Window-based training process with sliding model parts
• Dynamic tensor selection under coordinated runtime budget
• Harmonized local-global tensor importance adjustment module