A Memory Efficient Unified Algorithm for Online Learning of Linear Dynamical Systems

2026-07-02

AI Summary

This work addresses the challenge of balancing memory efficiency and theoretical guarantees in online prediction of unknown linear dynamical systems. The authors propose a unified online learning algorithm whose memory complexity depends only on the number $k$ of unstable modes of the system, rather than its full state dimension. By identifying $k$ as the intrinsic measure of complexity, they construct a predictor requiring only $\tilde{O}(k)$ learnable parameters—the first to do so—and establish a theoretical lower bound by proving that any filter-based method necessitates at least $k$ filters. Empirical results demonstrate that, in high-dimensional settings, the algorithm significantly outperforms existing approaches under the same parameter budget, while enjoying sublinear regret, memory efficiency, and robustness to non-diagonalizable, complex, and explosive modes.

Abstract

Motivated by the challenge of stabilizing a general unknown linear dynamical system (LDS) from observations, we study the natural prerequisite of online prediction. Our goal is to achieve sublinear regret with a memory footprint that adapts to the intrinsic complexity of the dynamics rather than the full hidden -- state dimension. We focus on the practically central regime of systems with low instability complexity -- eigenvalues outside the real stable interval that do not decay rapidly, together with non-semisimple modes-potentially embedded in an otherwise stable real spectrum of much higher dimension; we write $k$ for this count. This regime is the primary setting in which stabilization is plausible: we show that many systems with high instability complexity cannot be stabilized without exponentially large controls. Thus, prediction is meaningful for stabilization precisely when the instability complexity is small. Within this regime, we introduce a unified online algorithm that handles every LDS (including non-diagonalizable systems with complex or exploding modes) with a learnable parameter count of $\widetilde{O}(k)$. Finally, we prove a lower bound showing that $k$ is a valid complexity measure: any filter-based predictor needs at least $k$ filters. Experiments corroborate our theory: on a high-dimensional system, our predictor sharply outperforms prior methods at an equal parameter budget.

Problem

• linear dynamical systems
• online learning
• memory efficiency
• instability complexity
• sublinear regret

Innovation

• linear dynamical systems
• online learning
• memory efficiency
• instability complexity
• sublinear regret