Bandwidth-Aware Network Topology Optimization for Decentralized Learning

2025-12-08

AI Summary

In decentralized learning, existing network topology design methods neglect bandwidth constraints, leading to slow consensus convergence. Method: This paper proposes a bandwidth-aware topology optimization framework that maximizes algebraic connectivity under an edge-cardinality constraint, formulated as a Mixed-Integer Semidefinite Program (MISDP). We introduce a novel maximum-bandwidth allocation strategy and design an efficient solver integrating the Alternating Direction Method of Multipliers (ADMM) with the Conjugate Gradient method, enabling scalable optimization in heterogeneous bandwidth environments. Contribution/Results: Experiments on real-world datasets demonstrate that the optimized topologies accelerate consensus by 1.11× (homogeneous bandwidth) and 1.21× (heterogeneous bandwidth) over baseline methods, significantly improving the convergence speed of distributed training.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Network topology is critical for efficient parameter synchronization in distributed learning over networks. However, most existing studies do not account for bandwidth limitations in network topology design. In this paper, we propose a bandwidth-aware network topology optimization framework to maximize consensus speed under edge cardinality constraints. For heterogeneous bandwidth scenarios, we introduce a maximum bandwidth allocation strategy for the edges to ensure efficient communication among nodes. By reformulating the problem into an equivalent Mixed-Integer SDP problem, we leverage a computationally efficient ADMM-based method to obtain topologies that yield the maximum consensus speed. Within the ADMM substep, we adopt the conjugate gradient method to efficiently solve large-scale linear equations to achieve better scalability. Experimental results demonstrate that the resulting network topologies outperform the benchmark topologies in terms of consensus speed, and reduce the training time required for decentralized learning tasks on real-world datasets to achieve the target test accuracy, exhibiting speedups of more than $1.11 imes$ and $1.21 imes$ for homogeneous and heterogeneous bandwidth settings, respectively.

Problem

• Optimizes network topology for decentralized learning under bandwidth constraints
• Maximizes consensus speed with edge cardinality and bandwidth allocation
• Solves mixed-integer SDP via ADMM to improve training efficiency

Innovation

• Bandwidth-aware topology optimization for decentralized learning
• Maximum bandwidth allocation strategy for heterogeneous scenarios
• ADMM-based method with conjugate gradient for scalability