Higher-Order Equilibrium Tracking for EM-Compressible Online Estimation

2026-05-09

AI Summary

This work addresses the challenge of decoupling the empirical batch optimum from tracking error within finite time horizons in online estimation. By reframing online estimation as the tracking of a moving empirical equilibrium, the authors freeze the batch equilibrium and decompose lag terms to isolate error sources. They introduce a “batch-to-online transfer” theorem, leveraging structural conditions—namely EM-compressibility and EM-jet^R-compressibility—together with a high-order equilibrium jet predictor, a frozen corrector, and streaming statistical compression techniques, enabling efficient evaluation of equilibrium responses and Newton corrections. In the context of latent-variable linear Gaussian covariance estimation, the method achieves a local tracking rate of O(T^{−ν(m+1)}), yielding explicit finite-sample risk bounds and a certified restart rule.

Abstract

We study online estimation in latent-variable models by recasting the problem as tracking a moving empirical equilibrium. Standard online EM and stochastic approximation analyses primarily study convergence toward the population parameter and typically do not isolate the empirical batch optimum from the online tracking error at finite horizon. Our framework decomposes the online estimate into the frozen batch equilibrium at the current running statistic and a tracking lag that captures the algorithm's delay behind this moving target. We prove a batch-to-online transfer theorem: provided $\lVert e_T \rVert_{L^{2}} = o(T^{-1/2})$, the online estimator inherits the batch central limit theorem and the sharp first-order risk constant. Our key observation is that the empirical optimum evolves on a smooth equilibrium manifold indexed by the running statistic. An $m$-th order equilibrium-jet predictor combined with an order-$ν$ frozen corrector yields localized tracking rates $O(T^{-ν(m+1)})$. We formalize EM-compressibility and EM-jet$^R$-compressibility as the structural conditions that make the equilibrium response and the Newton corrector evaluable from a retained streaming statistic. The theory is instantiated in latent linear Gaussian covariance estimation, where the first-order scheme operates on a compressed $d \times d$ statistic with explicit finite-sample risk envelopes and a certified restart rule.

Problem

• online estimation
• empirical equilibrium
• tracking error
• latent-variable models
• EM-compressibility

Innovation

• online EM
• equilibrium tracking
• EM-compressibility
• higher-order predictor
• batch-to-online transfer