Robust and Efficient Collaborative Learning

2025-10-09

AI Summary

Collaborative machine learning in decentralized settings faces dual challenges: adversarial behavior and high communication overhead. To address these, we propose RPEL—a robust, serverless decentralized training framework that integrates pull-based parameter propagation with epidemic learning. RPEL enables asynchronous updates via stochastic subset parameter pulling and robust aggregation. Theoretically, we prove that RPEL converges with high probability under adversarial conditions; its communication complexity improves significantly from O(n²) to O(n log n). Empirically, RPEL maintains accuracy comparable to fully connected architectures across diverse adversarial attacks and demonstrates superior scalability and robustness as network size increases. Our key contribution is the first principled integration of pull-based mechanisms with epidemic propagation strategies—thereby breaking the traditional efficiency–robustness trade-off inherent in decentralized learning.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Collaborative machine learning is challenged by training-time adversarial behaviors. Existing approaches to tolerate such behaviors either rely on a central server or induce high communication costs. We propose Robust Pull-based Epidemic Learning (RPEL), a novel, scalable collaborative approach to ensure robust learning despite adversaries. RPEL does not rely on any central server and, unlike traditional methods, where communication costs grow in $mathcal{O}(n^2)$ with the number of nodes $n$, RPEL employs a pull-based epidemic-based communication strategy that scales in $mathcal{O}(n log n)$. By pulling model parameters from small random subsets of nodes, RPEL significantly lowers the number of required messages without compromising convergence guarantees, which hold with high probability. Empirical results demonstrate that RPEL maintains robustness in adversarial settings, competes with all-to-all communication accuracy, and scales efficiently across large networks.

Problem

• Tolerating training-time adversarial behaviors collaboratively
• Reducing communication costs in decentralized machine learning
• Ensuring robust learning without central server reliance

Innovation

• RPEL uses pull-based epidemic communication strategy
• RPEL eliminates central server dependency completely
• RPEL scales with O(n log n) communication complexity