Bayesian Robust Aggregation for Federated Learning

2025-05-05

AI Summary

Federated learning faces robustness challenges from malicious clients dynamically submitting tampered model updates, with unknown and time-varying numbers of compromised participants. To address this, we propose a Bayesian inference–based adaptive robust aggregation method: it marginalizes over the prior distribution of honest clients and estimates the optimal global model via maximum marginal likelihood—without requiring prior knowledge of the number of corrupted clients. This work is the first to introduce marginal likelihood maximization into federated aggregation, achieving both the simplicity of FedAvg and the attack resilience of specialized robust methods (e.g., Krum), while natively supporting time-varying attack scales. Evaluated on three image classification benchmarks under both static and dynamic attack settings, our method achieves state-of-the-art performance, significantly outperforming FedAvg, Krum, and GeoMed.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated Learning enables collaborative training of machine learning models on decentralized data. This scheme, however, is vulnerable to adversarial attacks, when some of the clients submit corrupted model updates. In real-world scenarios, the total number of compromised clients is typically unknown, with the extent of attacks potentially varying over time. To address these challenges, we propose an adaptive approach for robust aggregation of model updates based on Bayesian inference. The mean update is defined by the maximum of the likelihood marginalized over probabilities of each client to be `honest'. As a result, the method shares the simplicity of the classical average estimators (e.g., sample mean or geometric median), being independent of the number of compromised clients. At the same time, it is as effective against attacks as methods specifically tailored to Federated Learning, such as Krum. We compare our approach with other aggregation schemes in federated setting on three benchmark image classification data sets. The proposed method consistently achieves state-of-the-art performance across various attack types with static and varying number of malicious clients.

Problem

• Federated Learning is vulnerable to adversarial attacks from corrupted clients
• The number of compromised clients is typically unknown and dynamic
• Existing methods lack robustness and adaptability to varying attack scenarios

Innovation

• Bayesian inference for robust aggregation
• Adaptive method against adversarial attacks
• Independent of compromised clients count