SpectralKrum: A Spectral-Geometric Defense Against Byzantine Attacks in Federated Learning

2025-12-12

AI Summary

To address the failure of robust aggregation under Byzantine client directional attacks in federated learning with non-IID data, this paper proposes a novel robust aggregation method integrating spectral subspace estimation and geometric neighborhood selection. We first model historical model updates as lying on a low-dimensional manifold, enabling dynamic, auxiliary-data-free defense via projection-based compression and adaptive thresholding of orthogonal residual energy. The method unifies Krum’s geometric selection with subspace-structural constraints to enhance resilience. Extensive evaluation—over 56,000 rounds on CIFAR-10 under highly skewed non-IID settings (α = 0.1), covering eight baselines and seven attack types—demonstrates significant improvements over state-of-the-art methods. Notably, it maintains high accuracy and stability against sophisticated, stealthy directional attacks such as adaptive-steer and buffer-drift, where existing approaches degrade substantially.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Federated Learning (FL) distributes model training across clients who retain their data locally, but this architecture exposes a fundamental vulnerability: Byzantine clients can inject arbitrarily corrupted updates that degrade or subvert the global model. While robust aggregation methods (including Krum, Bulyan, and coordinate-wise defenses) offer theoretical guarantees under idealized assumptions, their effectiveness erodes substantially when client data distributions are heterogeneous (non-IID) and adversaries can observe or approximate the defense mechanism. This paper introduces SpectralKrum, a defense that fuses spectral subspace estimation with geometric neighbor-based selection. The core insight is that benign optimization trajectories, despite per-client heterogeneity, concentrate near a low-dimensional manifold that can be estimated from historical aggregates. SpectralKrum projects incoming updates into this learned subspace, applies Krum selection in compressed coordinates, and filters candidates whose orthogonal residual energy exceeds a data-driven threshold. The method requires no auxiliary data, operates entirely on model updates, and preserves FL privacy properties. We evaluate SpectralKrum against eight robust baselines across seven attack scenarios on CIFAR-10 with Dirichlet-distributed non-IID partitions (alpha = 0.1). Experiments spanning over 56,000 training rounds show that SpectralKrum is competitive against directional and subspace-aware attacks (adaptive-steer, buffer-drift), but offers limited advantage under label-flip and min-max attacks where malicious updates remain spectrally indistinguishable from benign ones.

Problem

• Defends against Byzantine attacks in federated learning
• Handles non-IID client data and adaptive adversaries
• Uses spectral subspace estimation and geometric selection

Innovation

• Uses spectral subspace estimation for benign trajectory identification
• Applies Krum selection in compressed coordinate space
• Filters updates via data-driven orthogonal residual threshold