🤖 AI Summary
This paper addresses online control of dynamic systems under adversarial environments—where cost functions and model disturbances are arbitrarily chosen by an adversary—seeking low regret relative to a benchmark policy class, rather than relying on post-hoc optimality or stochastic noise assumptions as in classical optimal or robust control. We propose the new paradigm of *online nonstochastic control*, which systematically integrates online convex optimization and convex relaxation into the classical control framework for the first time. Our approach abandons probabilistic modeling and instead establishes deterministic, regret-based performance guarantees, coupled with iterative optimization algorithms. We derive finite-time regret bounds and computational complexity analyses. Theoretically, our algorithm achieves provably low regret—$O(sqrt{T})$—while remaining efficiently implementable within standard control settings.
📝 Abstract
This text presents an introduction to an emerging paradigm in control of dynamical systems and differentiable reinforcement learning called online nonstochastic control. The new approach applies techniques from online convex optimization and convex relaxations to obtain new methods with provable guarantees for classical settings in optimal and robust control. The primary distinction between online nonstochastic control and other frameworks is the objective. In optimal control, robust control, and other control methodologies that assume stochastic noise, the goal is to perform comparably to an offline optimal strategy. In online nonstochastic control, both the cost functions as well as the perturbations from the assumed dynamical model are chosen by an adversary. Thus the optimal policy is not defined a priori. Rather, the target is to attain low regret against the best policy in hindsight from a benchmark class of policies. This objective suggests the use of the decision making framework of online convex optimization as an algorithmic methodology. The resulting methods are based on iterative mathematical optimization algorithms, and are accompanied by finite-time regret and computational complexity guarantees.
