On the Approximability of Stationary Processes using the ARMA Model

2024-08-20

arXiv.org

AI Summary

Existing research on ARMA approximation of stationary stochastic processes lacks rigorous theoretical foundations and precise error characterizations. Method: This work adopts a novel analytical framework centered on the process generating function—rather than the conventional spectral measure—and employs the sup-norm on the unit circle to quantify approximation error. It introduces the spectral lemma for quantitative error analysis of ARMA approximation, transforming heuristic rational approximation arguments into rigorous theorems. Contribution/Results: We prove that Padé approximants are not universally optimal for fixed-order ARMA approximation and construct explicit counterexamples of non-approximable stationary processes. Furthermore, we characterize the class of stationary processes admitting ARMA approximation, derive exact asymptotic approximation bounds for canonical processes (e.g., AR, MA, fractional Gaussian noise), and extend beyond traditional prediction-error-centric paradigms to establish a comprehensive theory of spectral approximation fidelity.

Technology Category

• Machine Learning: Learning Theory
• Reasoning under Uncertainty: Stochastic Optimization
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Within the theoretical literature on stationary random variables, pure Moving Average models and pure Autoregressive models have a rich body of work, but the corresponding literature on Autoregressive Moving Average (ARMA) models is very sparse. We attempt to fill certain gaps in this sparse line of work. Central to our observations is the spectral lemma connecting supnorm based function approximation on the unit circle to random variable approximation. This method allows us to provide quantitative approximation bounds in contrast with the qualitative boundedness and stability guarantees associated with unit root tests. Using the spectral lemma we first identify a class of stationary processes where approximation guarantees are feasible. This turns a known heuristic argument motivating ARMA models based on rational approximations into a rigorous result. Second, we identify an idealized stationary random process for which we conjecture that a good ARMA approximation is not possible. Third, we calculate exact approximation bounds for an example process, and a constructive proof that, for a given order, Pad'e approximations do not always correspond to the best ARMA approximation. Unlike prior literature, our approach uses the generating function of the random process rather than the spectral measure, and further our results focus on approximation error of the random variable rather than the prediction error as in some classical infimum results by Szego, Kolmogorov, and Wiener.

Problem

• Filling gaps in ARMA model literature for stationary processes.
• Providing quantitative approximation bounds using spectral lemma.
• Identifying processes where ARMA approximation is feasible or impossible.

Innovation

• Uses spectral lemma for approximation bounds
• Focuses on generating function, not spectral measure
• Calculates exact bounds for ARMA approximation