Free Parametrization of L2-bounded State Space Models

2025-03-31

AI Summary

Structured State Space Models (SSMs) face fundamental limitations in system identification and optimal control due to challenges in guaranteeing stability and robustness. This paper proposes L2RU, the first SSM architecture enabling fully unconstrained parameterization: it explicitly parametrizes a non-conservative, analytically differentiable upper bound on the L₂ gain of the underlying discrete-time linear time-invariant (LTI) system, thereby ensuring input-output L₂ stability and robustness for *all* parameter values. Critically, L2RU eliminates all explicit stability constraints—enabling end-to-end gradient-based optimization without projection or penalty terms—and introduces a tailored initialization strategy for long-sequence modeling. Experiments demonstrate that L2RU significantly outperforms existing baselines on system identification benchmarks while provably maintaining stability. By unifying theoretical safety guarantees with practical trainability, L2RU establishes a new paradigm for learning-based control.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Structured state-space models (SSMs) have emerged as a powerful architecture in machine learning and control, featuring stacked layers where each consists of a linear time-invariant (LTI) discrete-time system followed by a nonlinearity. While SSMs offer computational efficiency and excel in long-sequence predictions, their widespread adoption in applications like system identification and optimal control is hindered by the challenge of ensuring their stability and robustness properties. We introduce L2RU, a novel parametrization of SSMs that guarantees input-output stability and robustness by enforcing a prescribed L-bound for all parameter values. This design eliminates the need for complex constraints, allowing unconstrained optimization over L2RUs by using standard methods such as gradient descent. Leveraging tools from system theory and convex optimization, we derive a non-conservative parametrization of square discrete-time LTI systems with a specified L2-bound, forming the foundation of the L2RU architecture. Additionally, we enhance its performance with a bespoke initialization strategy optimized for long input sequences. Through a system identification task, we validate L2RU's superior performance, showcasing its potential in learning and control applications.

Problem

• Ensuring stability and robustness in structured state-space models
• Eliminating complex constraints for unconstrained optimization
• Improving performance in long-sequence system identification tasks

Innovation

• L2RU ensures L2-bound stability
• Unconstrained optimization via gradient descent
• Custom initialization for long sequences