On the Space Complexity of Online Convolution

2025-04-30

AI Summary

This paper investigates the space complexity of online convolution: given a dynamic input stream and a lower-triangular Toeplitz matrix (T), what is the minimal memory required to compute the convolution output in real time? The core method establishes a tight correspondence between the rationality of the generating function of (T)’s first column and the algorithm’s space complexity: for a degree-(d) rational generating function, we prove (Theta(d)) space bounds; for irrational generating functions (e.g., (1/sqrt{1-x})), we show an (Omega(t)) time-dependent lower bound and unbounded space requirement. Our analysis integrates algebraic techniques, generating function theory, linear system modeling, and information-theoretic lower-bound proofs. This work provides the first complete classification framework for the space complexity of online convolution, yielding rigorous memory bottleneck characterizations essential for streaming algorithms—such as differentially private continuous counting—where storage efficiency is critical.

Technology Category

• Machine Learning: Online Learning & Bandits
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

We study a discrete convolution streaming problem. An input arrives as a stream of numbers $z = (z_0,z_1,z_2,ldots)$, and at time $t$ our goal is to output $(Tz)_t$ where $T$ is a lower-triangular Toeplitz matrix. We focus on space complexity; the algorithm can store a buffer of $eta(t)$ numbers in order to achieve this goal. We characterize space complexity when algorithms perform algebraic operations. The matrix $T$ corresponds to a generating function $G(x)$. If $G(x)$ is rational of degree $d$, then it is known that the space complexity is at most $O(d)$. We prove a corresponding lower bound; the space complexity is at least $Omega(d)$. In addition, for irrational $G(x)$, we prove that the space complexity is infinite. We also provide finite-time guarantees. For example, for the generating function $frac{1}{sqrt{1-x}}$ that was studied in various previous works in the context of differentially private continual counting, we prove a sharp lower bound on the space complexity; at time $t$, it is at least $Omega(t)$.

Problem

• Studies space complexity of online convolution streaming problem
• Proves lower bound Ω(d) for rational generating functions
• Shows infinite space needed for irrational generating functions

Innovation

• Focuses on space complexity in streaming algorithms
• Uses lower-triangular Toeplitz matrix for convolution
• Proves bounds for rational and irrational generating functions