Approximate Distributed Coded Computing: Polynomial Codes and Randomized Sketching

2026-05-15

AI Summary

This work addresses the tail latency bottleneck in large-scale distributed computing caused by slow or unresponsive nodes by proposing a novel framework that integrates coded computation with randomized numerical linear algebra. By systematically combining polynomial codes and random sketching techniques for the first time, and incorporating a probabilistic compression mechanism, the approach significantly enhances computational efficiency while preserving fault tolerance. The method effectively reduces both communication and computational overhead in high-dimensional machine learning tasks, accelerating model training without compromising robustness. This solution offers a theoretically rigorous and practically efficient paradigm for distributed optimization.

Abstract

Coded computing is a distributed paradigm that uses coding theory to introduce \textit{redundancy} and overcome bottlenecks in large-scale systems. In the same vein, randomized numerical linear algebra employs probabilistic methods to \textit{compress} and accelerate linear algebraic operations, addressing challenges in high-dimensional data analysis. This article reviews the foundations of both fields and presents distributed schemes that combine techniques from both to speed up optimization and machine learning algorithms, in the presence of slow or non-responsive servers. Along the way, we touch on various related topics and mathematical concepts.

Problem

• coded computing
• randomized numerical linear algebra
• distributed optimization
• machine learning
• straggler mitigation

Innovation

• coded computing
• randomized numerical linear algebra
• polynomial codes
• randomized sketching
• distributed optimization