Optimality in Decentralized Optimization under Bandwidth Constraints

2026-03-21

AI Summary

This work addresses the lack of tight and practical time complexity bounds in bandwidth-constrained decentralized non-convex stochastic optimization. By introducing graph-theoretic concepts—specifically the min-cut/max-flow quantity, Gomory-Hu trees, and Steiner tree packing—we develop a bandwidth-aware communication-computation co-analysis framework and derive a near-optimal lower bound on time complexity. Building on this foundation, we propose two algorithms: Grace SGD for homogeneous settings and Leon SGD for heterogeneous environments. Both algorithms achieve time complexity that is optimal up to logarithmic factors in their respective scenarios, significantly outperforming existing methods.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We consider a realistic decentralized setup with bandwidth-constrained communication and derive optimal time complexities for non-convex stochastic parallel and asynchronous optimization (up to logarithmic factors). We develop the corresponding methods, Grace SGD and Leon SGD, for both homogeneous and heterogeneous settings. Unlike previous work, our optimal bounds are characterized in terms of min-cut/max-flow quantities and rely on tools from Gomory-Hu trees and Steiner Tree Packing problems, providing tighter and more practical complexities.

Problem

• decentralized optimization
• bandwidth constraints
• non-convex optimization
• time complexity
• stochastic optimization

Innovation

• decentralized optimization
• bandwidth constraints
• optimal complexity
• Gomory-Hu trees
• non-convex stochastic optimization