🤖 AI Summary
This work investigates the fundamental computation-communication trade-off for distributed linearly separable computation under cyclic data allocation, specifically in the straggler-resilient setting where the master node tolerates arbitrary $N_r$ slow or unresponsive workers. To address performance bottlenecks caused by stragglers—common in applications such as gradient coding and real-time rendering—we pioneer the application of interference alignment to this framework. Leveraging cyclic data placement and linear separability of the target function, we construct a near-optimal distributed encoding scheme. Theoretically, our scheme achieves order-optimal performance: it simultaneously approaches the information-theoretic lower bounds on both computation load (number of data subsets assigned per worker) and communication cost (number of coded messages the master must collect). This resolves, for the first time, a long-standing open problem on order optimality under cyclic allocation and straggler resilience, significantly outperforming existing approaches.
📝 Abstract
Distributed Linearly Separable Computation problem under the cyclic assignment is studied in this paper. It is a problem widely existing in cooperated distributed gradient coding, real-time rendering, linear transformers, etc. In a distributed computing system, a master asks N distributed workers to compute a linearly separable function from K datasets. The task function can be expressed as Kc linear combinations of K messages, where each message is the output of one individual function of one dataset. Straggler effect is also considered, such that from the answers of each Nr worker, the master should recover the task. The computation cost is defined as the number of datasets assigned to each worker, while the communication cost is defined as the number of (coded) messages which should be received. The objective is to characterize the optimal tradeoff between the computation and communication costs. Various distributed computing scheme were proposed in the literature with a well-known cyclic data assignment, but the (order) optimality of this problem remains open, even under the cyclic assignment. This paper proposes a new computing scheme with the cyclic assignment based on interference alignment, which is near optimal under the cyclic assignment.
