The Impact of Approximation on Algorithmic Progress

2026-04-30

AI Summary

This study addresses the computational bottlenecks in scientific computing arising from the infeasibility of exact algorithms for large-scale problems. Through a systematic evaluation of approximation methods across 118 core algorithmic problems—integrating complexity analysis, taxonomies of approximation algorithms, and historical context—the work presents the first large-scale empirical evidence demonstrating that only approximately 20% of these problems derive substantial benefit from approximation. Notably, one-quarter of exponential-time-hard problems admit polynomial-time approximation schemes, and the adoption of approximation strategies increases the proportion of linear-time solvable problems by 23%. By quantifying the trade-offs between accuracy and efficiency, this research offers theoretical insights to guide the design of AI-driven and high-performance algorithms.

Abstract

In nearly every discipline, scientific computations are limited by the cost and speed of computation. For example, the best-known exact algorithms for the canonical Traveling Salesman Problem would take centuries to run on an instance of size 1 million. A natural response to such limits is to try to find new algorithms or to parallelize existing ones, but many algorithms are already at their theoretically-optimal level and parallelization is often impossible or prohibitively expensive. Starting in the 1960's, computer scientists pursued another solution: allowing solutions to have a small amount of error (i.e. approximating them). In this paper, we survey 118 of the most important algorithm problems in computer science, quantifying the gains and tradeoffs from approximation that have been discovered over the history of the field. Overall, only $\approx$20\% of problems have benefited from approximation. However, those with good approximate algorithms can be dramatically faster to compute with little cost to accuracy. For example, a quarter of computationally intractable problems (e.g. those that take exponential time to compute) have polynomial time approximate algorithms. Approximation also increases the number of algorithms that can run in linear time by 23\%, opening up new computational opportunities for those working in the big data regime. This work also sheds light on what should be expected from progress in AI, where approximation is at the heart of how deep learning works.

Problem

• approximation
• algorithmic progress
• computational complexity
• intractable problems
• big data

Innovation

• approximation algorithms
• computational complexity
• algorithmic tradeoffs
• big data
• AI progress