Byzantine-resilient federated online learning for Gaussian process regression

2025-07-18

AI Summary

This paper addresses the vulnerability of Gaussian Process Regression (GPR) to Byzantine failures in federated online learning. To this end, we propose a Byzantine-resilient collaborative modeling framework. Methodologically, we design a robust aggregation rule based on the Product of Experts (PoE), enabling online collaborative learning across three layers: local GPR models at edge agents, cloud-aggregated GPR at the central server, and client-side fused GPR. We theoretically characterize the performance gain boundary of the robust aggregation for local models. Our key contributions are: (i) the first integration of PoE into Byzantine-robust federated GPR, achieving a balance between statistical efficiency and resilience against malicious attacks; and (ii) empirical validation on synthetic and two real-world medium-scale datasets, demonstrating significant improvements in prediction accuracy and system robustness—particularly maintaining stable convergence even under 20% Byzantine node participation.

Technology Category

• Machine Learning: Online Learning & Bandits
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we study Byzantine-resilient federated online learning for Gaussian process regression (GPR). We develop a Byzantine-resilient federated GPR algorithm that allows a cloud and a group of agents to collaboratively learn a latent function and improve the learning performances where some agents exhibit Byzantine failures, i.e., arbitrary and potentially adversarial behavior. Each agent-based local GPR sends potentially compromised local predictions to the cloud, and the cloud-based aggregated GPR computes a global model by a Byzantine-resilient product of experts aggregation rule. Then the cloud broadcasts the current global model to all the agents. Agent-based fused GPR refines local predictions by fusing the received global model with that of the agent-based local GPR. Moreover, we quantify the learning accuracy improvements of the agent-based fused GPR over the agent-based local GPR. Experiments on a toy example and two medium-scale real-world datasets are conducted to demonstrate the performances of the proposed algorithm.

Problem

• Develop Byzantine-resilient federated GPR algorithm
• Improve learning under adversarial agent behavior
• Quantify accuracy gains in fused GPR models

Innovation

• Byzantine-resilient federated GPR algorithm
• Product of experts aggregation rule
• Agent-based fused GPR refinement