Massively Parallel Minimum Spanning Tree in General Metric Spaces

2024-08-12

arXiv.org

AI Summary

This paper studies the massively parallel computation (MPC) of minimum spanning trees (MST) in general metric spaces under the strict sublinear-memory MPC model, where each machine has $O(n^delta)$ space. It breaks the long-standing $Omega(log n)$ round-complexity barrier. For arbitrary pairwise distances satisfying the triangle inequality, the authors present the first $(1+varepsilon)$-approximate MST algorithm requiring only $O(log(1/varepsilon) + loglog n)$ rounds—significantly improving upon the large-constant approximation of SODA’24. They further establish the first unconditional lower bound of $Omega(log(1/varepsilon))$ rounds for *any* MST algorithm (not restricted to component-stable ones), proving asymptotic optimality in $varepsilon$-precision. Technically, the approach integrates MPC graph algorithm design with hierarchical clustering in metric spaces, sparsification, and an iterative refinement framework.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Other Foundations of Machine Learning
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

We study the minimum spanning tree (MST) problem in the massively parallel computation (MPC) model. Our focus is particularly on the *strictly sublinear* regime of MPC where the space per machine is $O(n^delta)$. Here $n$ is the number of vertices and constant $delta in (0, 1)$ can be made arbitrarily small. The MST problem admits a simple and folklore $O(log n)$-round algorithm in the MPC model. When the weights can be arbitrary, this matches a conditional lower bound of $Omega(log n)$ which follows from a well-known 1vs2-Cycle conjecture. As such, much of the literature focuses on breaking the logarithmic barrier in more structured variants of the problem, such as when the vertices correspond to points in low- [ANOY14, STOC'14] or high-dimensional Euclidean spaces [JMNZ, SODA'24]. In this work, we focus more generally on metric spaces. Namely, all pairwise weights are provided and guaranteed to satisfy the triangle inequality, but are otherwise unconstrained. We show that for any $varepsilon>0$, a $(1+varepsilon)$-approximate MST can be found in $O(log frac{1}{varepsilon} + log log n)$ rounds, which is the first $o(log n)$-round algorithm for finding any constant approximation in this setting. Other than being applicable to more general weight functions, our algorithm also slightly improves the $O(log log n cdot log log log n)$ round-complexity of [JMNZ24, SODA'24] and significantly improves its approximation from a large constant to $1+varepsilon$. On the lower bound side, we prove that under the 1vs2-Cycle conjecture, $Omega(log frac{1}{varepsilon})$ rounds are needed for finding a $(1+varepsilon)$-approximate MST in general metrics. It is worth noting that while many existing lower bounds in the MPC model under the 1vs2-Cycle conjecture only hold against"component stable"algorithms, our lower bound applies to *all* algorithms.

Problem

• Solving minimum spanning tree in massively parallel computation model
• Achieving sublogarithmic rounds for approximate MST in metrics
• Providing tight lower bounds for MST approximation in MPC

Innovation

• Approximate MST in general metric spaces
• Rounds complexity of O(log 1/ε + log log n)
• Breaking logarithmic barrier for constant approximation