Quantized Online LQR

2026-04-13

AI Summary

This work addresses the challenge of online linear quadratic regulator (LQR) control for unknown dynamical systems under communication constraints. Conventional approaches suffer from persistent noise due to per-step state quantization and high communication overhead. To overcome these limitations, the authors propose a novel paradigm wherein the local agent estimates system dynamics from observations and transmits only this dynamic estimate over a low-rate communication link to a remote controller. The remote controller then computes the optimal policy and sends it back for execution using the local agent’s exact state. The study establishes, for the first time, an Ω(log T) fundamental lower bound on communication complexity required to achieve sublinear regret and introduces the QCE-LQR algorithm that matches this bound. As the quantization resolution increases, QCE-LQR’s regret smoothly converges to that of the unquantized certainty-equivalent benchmark. Empirical evaluations on four standard systems, including a Boeing 747 model, demonstrate that QCE-LQR attains performance comparable to the unquantized controller within T = 10,000 steps.

Technology Category

• Multiagent Systems: Agent Communication
• Search and Optimization: Learning to Search
• Machine Learning: Quantum Machine Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We study online linear-quadratic regulation (LQR) with unknown dynamics under communication rate constraints. Classical networked control quantizes the plant state at every time step, requiring $O(T)$ total bits while injecting persistent quantization noise that limits control performance. We consider a setting where the plant observes its state locally and can estimate system dynamics via ordinary least squares, while a remote controller possesses knowledge of the control cost. Rather than quantizing the raw state, the plant transmits learned dynamics estimates over a rate-limited uplink, and the controller returns the optimal control policy so that the plant can compute actions locally using its superior state knowledge. We first prove a fundamental information-theoretic lower bound: any scheme achieving $O(T^α)$ regret for $α\in [1/2,1)$ compared to the optimal infinite horizon LQR controller that knows the true system dynamics must transmit at least $Ω(\log T)$ bits. We then design the \textbf{Quantized Certainty Equivalent (QCE-LQR)} algorithm, which matches this bound. The resulting regret bound contains inflation factors $Q_{\mathrm{slow}}(\varrho)$ and $Q_{\mathrm{fast}}(\varrho)$ that vanish as the codebook resolution increases, smoothly recovering the unquantized baseline regret. Numerical experiments on four benchmark systems -- from a scalar unstable plant to a 24-parameter Boeing 747 lateral model -- confirm that a variant of QCE-LQR achieves regret comparable to an unquantized certainty equivalent controller over a horizon of $T=10{,}000$ steps.

Problem

• quantized control
• online LQR
• communication constraints
• unknown dynamics
• regret minimization

Innovation

• Quantized LQR
• Online Control
• Communication Constraints
• Certainty Equivalence
• Information-Theoretic Lower Bound