On the Optimality of Coded Distributed Computing for Ring Networks

2025-06-30

AI Summary

In distributed computing over a ring topology where each node can only communicate with neighbors within broadcast distance $d$, intermediate data exchange—particularly for MapReduce-like operations—creates a severe communication bottleneck. Method: This paper proposes a novel scheme integrating redundant computation (where each map function is computed in parallel by $r$ nodes) and coded transmission. It introduces a coding strategy based on consecutive reverse carpooling, combined with nearest-neighbor-first dissemination and cyclic data placement, and establishes optimality via information-theoretic lower bounds. Contributions/Results: First, it achieves the optimal trade-off among communication load, computation redundancy $r$, and broadcast distance $d$ on ring networks. Second, it theoretically proves that redundancy yields an *additive* gain while broadcast distance yields a *multiplicative* gain—breaking the conventional paradigm of uncoded repetition. Third, under $N gg d,r$, the scheme attains exact optimality for all-gather and asymptotic optimality for all-to-all under cyclic placement.

Technology Category

• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

We consider a coded distributed computing problem in a ring-based communication network, where $N$ computing nodes are arranged in a ring topology and each node can only communicate with its neighbors within a constant distance $d$. To mitigate the communication bottleneck in exchanging intermediate values, we propose new coded distributed computing schemes for the ring-based network that exploit both ring topology and redundant computation (i.e., each map function is computed by $r$ nodes). Two typical cases are considered: all-gather where each node requires all intermediate values mapped from all input files, and all-to-all where each node requires a distinct set of intermediate values from other nodes. For the all-gather case, we propose a new coded scheme based on successive reverse carpooling where nodes transmit every encoded packet containing two messages traveling in opposite directions along the same path. Theoretical converse proof shows that our scheme achieves the optimal tradeoff between communication load, computation load $r$, and broadcast distance $d$ when $Ngg d$. For the all-to-all case, instead of simply repeating our all-gather scheme, we delicately deliver intermediate values based on their proximity to intended nodes to reduce unnecessary transmissions. We derive an information-theoretic lower bound on the optimal communication load and show that our scheme is asymptotically optimal under the cyclic placement when $Ngg r$. The optimality results indicate that in ring-based networks, the redundant computation $r$ only leads to an additive gain in reducing communication load while the broadcast distance $d$ contributes to a multiplicative gain.

Problem

• Optimize communication in ring-based distributed computing networks
• Develop coded schemes for all-gather and all-to-all value exchanges
• Achieve optimal tradeoff between communication load and computation redundancy

Innovation

• Coded scheme with reverse carpooling for ring topology
• Optimal tradeoff between communication, computation, distance
• Asymptotically optimal delivery for all-to-all case