Warm Starting State-Space Models with Automata Learning

2026-03-05

AI Summary

This work addresses the data inefficiency of state space models (SSMs) in learning systems with symbolic structure and their difficulty in recovering the underlying state dynamics. It establishes, for the first time, a rigorous proof that Moore machines can be exactly represented by SSMs. Building on this result, the authors propose a hybrid approach that injects symbolic inductive bias into continuous learning by initializing SSMs with Moore machines learned via symbolic automata learning—encompassing both active and passive learning paradigms—as a warm start. Evaluated on the SYNTCOMP arbiter benchmark, this method substantially improves performance: warm-started SSMs converge 2–5 times faster than randomly initialized counterparts and achieve higher test accuracy.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We prove that Moore machines can be exactly realized as state-space models (SSMs), establishing a formal correspondence between symbolic automata and these continuous machine learning architectures. These Moore-SSMs preserve both the complete symbolic structure and input-output behavior of the original Moore machine, but operate in Euclidean space. With this correspondence, we compare the training of SSMs with both passive and active automata learning. In recovering automata from the SYNTCOMP benchmark, we show that SSMs require orders of magnitude more data than symbolic methods and fail to learn state structure. This suggests that symbolic structure provides a strong inductive bias for learning these systems. We leverage this insight to combine the strengths of both automata learning and SSMs in order to learn complex systems efficiently. We learn an adaptive arbitration policy on a suite of arbiters from SYNTCOMP and show that initializing SSMs with symbolically-learned approximations learn both faster and better. We see 2-5 times faster convergence compared to randomly initialized models and better overall model accuracies on test data. Our work lifts automata learning out of purely discrete spaces, enabling principled exploitation of symbolic structure in continuous domains for efficiently learning in complex settings.

Problem

• state-space models
• automata learning
• Moore machines
• inductive bias
• symbolic structure

Innovation

• state-space models
• Moore machines
• automata learning
• warm starting
• inductive bias