Topology-Aware Knowledge Propagation in Decentralized Learning

2025-05-16

AI Summary

This work addresses the low propagation efficiency of out-of-distribution (OOD) knowledge in decentralized learning under arbitrary communication topologies, revealing critical constraints imposed by topology structure and the spatial positioning of OOD-data nodes on knowledge diffusion. We propose the first topology-aware aggregation strategy, integrating graph signal processing with adaptive weighted averaging. By modeling OOD sensitivity and extracting local topological features, the method dynamically adapts to heterogeneous device connectivity. Unlike conventional topology-agnostic approaches, our framework explicitly leverages structural information to enhance global OOD knowledge dissemination. Extensive experiments across multiple models and diverse topologies demonstrate that our method improves OOD identification accuracy by an average of 123%, significantly outperforming state-of-the-art baselines.

Technology Category

• Search and Optimization: Distributed Search
• Machine Learning: Distributed Machine Learning & Federated Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems

Abstract

Decentralized learning enables collaborative training of models across naturally distributed data without centralized coordination or maintenance of a global model. Instead, devices are organized in arbitrary communication topologies, in which they can only communicate with neighboring devices. Each device maintains its own local model by training on its local data and integrating new knowledge via model aggregation with neighbors. Therefore, knowledge is propagated across the topology via successive aggregation rounds. We study, in particular, the propagation of out-of-distribution (OOD) knowledge. We find that popular decentralized learning algorithms struggle to propagate OOD knowledge effectively to all devices. Further, we find that both the location of OOD data within a topology, and the topology itself, significantly impact OOD knowledge propagation. We then propose topology-aware aggregation strategies to accelerate (OOD) knowledge propagation across devices. These strategies improve OOD data accuracy, compared to topology-unaware baselines, by 123% on average across models in a topology.

Problem

• Studying OOD knowledge propagation in decentralized learning
• Analyzing topology impact on OOD data distribution
• Proposing topology-aware aggregation to enhance OOD accuracy

Innovation

• Decentralized learning with arbitrary communication topologies
• Topology-aware aggregation for knowledge propagation
• Improved OOD data accuracy by 123%