Adaptive Federated Learning Defences via Trust-Aware Deep Q-Networks

2025-09-25

AI Summary

Federated learning (FL) suffers from poor robustness against poisoning and backdoor attacks in partially observable settings, where the server lacks full visibility into client behavior. Method: This paper introduces the first defense framework formulated as a Partially Observable Markov Decision Process (POMDP), proposing Trust-Aware Deep Q-Networks (TA-DQN). TA-DQN dynamically estimates and updates client trust scores by jointly fusing multi-source signals—including gradient norms, parameter update magnitudes, and model similarity—enabling long-horizon optimization of both robustness and accuracy. Contribution/Results: Unlike conventional static or heuristic defenses, TA-DQN achieves adaptive, sequential decision-making under uncertainty. Experiments on CIFAR-10 demonstrate that as client overlap increases, model accuracy improves while attack success rate drops significantly. Compared to baselines—including random selection, linear Q-networks, and policy gradient methods—TA-DQN attains superior Pareto-optimal trade-offs between robustness and classification accuracy.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Federated learning is vulnerable to poisoning and backdoor attacks under partial observability. We formulate defence as a partially observable sequential decision problem and introduce a trust-aware Deep Q-Network that integrates multi-signal evidence into client trust updates while optimizing a long-horizon robustness--accuracy objective. On CIFAR-10, we (i) establish a baseline showing steadily improving accuracy, (ii) show through a Dirichlet sweep that increased client overlap consistently improves accuracy and reduces ASR with stable detection, and (iii) demonstrate in a signal-budget study that accuracy remains steady while ASR increases and ROC-AUC declines as observability is reduced, which highlights that sequential belief updates mitigate weaker signals. Finally, a comparison with random, linear-Q, and policy gradient controllers confirms that DQN achieves the best robustness--accuracy trade-off.

Problem

• Defending federated learning from poisoning and backdoor attacks
• Optimizing long-term robustness-accuracy trade-off in partial observability
• Integrating multi-signal evidence for client trust updates

Innovation

• Trust-aware Deep Q-Network for client trust updates
• Multi-signal evidence integration in federated learning
• Long-horizon robustness-accuracy objective optimization