Simultaneous System Identification and Model Predictive Control with No Dynamic Regret

2024-07-04

arXiv.org

AI Summary

This work addresses nonlinear systems subject to unknown dynamics and external disturbances. Methodologically, it proposes an integrated online system identification and model predictive control (MPC) framework that combines reproducing kernel Hilbert space (RKHS) modeling, random Fourier feature approximation, online least-squares parameter adaptation, and learning-based receding-horizon MPC—compatible with control-affine structures. The approach achieves sublinear dynamic regret against an adversarial clairvoyant controller for the first time, while ensuring finite-time near-optimality and asymptotic convergence to optimality. To jointly handle modeling errors and exogenous disturbances, it introduces self-supervised learning and state- and input-adaptive disturbance modeling. Extensive validation is conducted on an inverted pendulum, quadrotor simulation, and real-world quadrotor hardware under challenging conditions—including wind gusts, ground effect, and aerodynamic drag—demonstrating robustness and high-precision trajectory tracking performance.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We provide an algorithm for the simultaneous system identification and model predictive control of nonlinear systems. The algorithm has finite-time near-optimality guarantees and asymptotically converges to the optimal (non-causal) controller. Particularly, the algorithm enjoys sublinear dynamic regret, defined herein as the suboptimality against an optimal clairvoyant controller that knows how the unknown disturbances and system dynamics will adapt to its actions. The algorithm is self-supervised and applies to control-affine systems with unknown dynamics and disturbances that can be expressed in reproducing kernel Hilbert spaces. Such spaces can model external disturbances and modeling errors that can even be adaptive to the system's state and control input. For example, they can model wind and wave disturbances to aerial and marine vehicles, or inaccurate model parameters such as inertia of mechanical systems. The algorithm first generates random Fourier features that are used to approximate the unknown dynamics or disturbances. Then, it employs model predictive control based on the current learned model of the unknown dynamics (or disturbances). The model of the unknown dynamics is updated online using least squares based on the data collected while controlling the system. We validate our algorithm in both hardware experiments and physics-based simulations. The simulations include (i) a cart-pole aiming to maintain the pole upright despite inaccurate model parameters, and (ii) a quadrotor aiming to track reference trajectories despite unmodeled aerodynamic drag effects. The hardware experiments include a quadrotor aiming to track a circular trajectory despite unmodeled aerodynamic drag effects, ground effects, and wind disturbances.

Problem

• Simultaneous system identification and control for nonlinear systems
• Sublinear dynamic regret against clairvoyant optimal controller
• Handling unknown disturbances and adaptive dynamics in control-affine systems

Innovation

• Simultaneous system identification and model predictive control
• Self-supervised algorithm with sublinear dynamic regret
• Random Fourier features for approximating unknown dynamics