Hybrid Federated Learning for Noise-Robust Training

2026-01-08

arXiv.org

AI Summary

This work addresses the poor robustness of conventional federated learning under noisy conditions and the slow convergence of federated distillation by proposing a hybrid federated learning framework. The framework enables user devices to flexibly upload either gradients or logits, while a base station dynamically adjusts the weighting between federated learning and federated distillation updates. To fully exploit the degrees of freedom in this hybrid design, the authors introduce an adaptive device clustering strategy based on Jenks optimization and a damping Newton method-driven weight selection mechanism. Experimental results demonstrate that the proposed approach significantly improves test accuracy under low signal-to-noise ratio conditions, thereby validating its effectiveness for noise-robust federated training.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated learning (FL) and federated distillation (FD) are distributed learning paradigms that train UE models with enhanced privacy, each offering different trade-offs between noise robustness and learning speed. To mitigate their respective weaknesses, we propose a hybrid federated learning (HFL) framework in which each user equipment (UE) transmits either gradients or logits, and the base station (BS) selects the per-round weights of FL and FD updates. We derive convergence of HFL framework and introduce two methods to exploit degrees of freedom (DoF) in HFL, which are (i) adaptive UE clustering via Jenks optimization and (ii) adaptive weight selection via a damped Newton method. Numerical results show that HFL achieves superior test accuracy at low SNR when both DoF are exploited.

Problem

• Federated Learning
• Noise Robustness
• Learning Speed
• Federated Distillation
• Hybrid Federated Learning

Innovation

• Hybrid Federated Learning
• Noise Robustness
• Adaptive Weight Selection
• Jenks Optimization
• Federated Distillation