Enhanced Byzantine-Robust Federated Learning Via Truncated-Quadratic Loss for Heterogeneous Data

2026-07-12

AI Summary

This work addresses the performance degradation of existing robust aggregation methods in federated learning under highly heterogeneous data distributions and a large number of Byzantine clients, where such methods often suffer from significant bias. To mitigate this issue, we propose a novel aggregation rule that introduces the truncated quadratic (TQ) loss into robust aggregation for the first time, optimizing a non-convex objective to effectively reduce bias. Leveraging convex conjugate theory, we establish the equivalence among existing approaches and design a TQ-based aggregator coupled with a bias estimation strategy, enabling efficient distributed training while providing order-optimal Byzantine robustness guarantees and adaptive estimation of the number of adversarial clients. Extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10 demonstrate that our method consistently outperforms state-of-the-art techniques across various attack scenarios and heterogeneity settings.

Abstract

Federated learning distributes data among $n$ clients, making it vulnerable to malicious attacks and data heterogeneity, which together pose challenges for robust learning. To tackle this issue, centered clipping and Huber aggregators have been exploited for Byzantine robustness. In this paper, we first demonstrate their equivalence via convex conjugate theory, and show that they can yield biased solutions in the presence of outliers, leading to failure under high data heterogeneity and a substantial fraction of outliers. Next, we propose a new robust aggregation rule that utilizes the truncated-quadratic (TQ) loss, effectively mitigating the biases of existing methods, such as centered clipping and Huber aggregators. We show that our aggregator achieves order-optimal Byzantine-robust learning under nonconvex loss functions and heterogeneous data, ultimately enhancing the reliability of federated learning systems. Additionally, we provide a robust deviation estimation strategy for TQ, demonstrating its effectiveness. Furthermore, we show that TQ maintains robustness even when only an estimate of the number of Byzantine clients is available. Finally, experimental results on MNIST, Fashion-MNIST, and CIFAR-10, indicate that our aggregator provides better robustness performance than the competing techniques.

Problem

• Byzantine-robust
• federated learning
• data heterogeneity
• outliers
• robust aggregation

Innovation

• Truncated-Quadratic loss
• Byzantine-robust aggregation
• Federated learning
• Data heterogeneity
• Nonconvex optimization