Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention

2026-03-15

AI Summary

This work proposes an unsupervised method that integrates causal representation learning with a locally sparse attention mechanism to identify disentangled parameters of dynamical systems directly from raw trajectory data, without requiring structural priors. While traditional system identification relies on predefined function libraries and deep learning models often lack interpretability and disentanglement, the proposed approach establishes novel identifiability theorems and graphical criteria, theoretically proving—for the first time—the necessity of local state-dependent causal structures for achieving full disentanglement and elucidating their relationship to global causal structures. Leveraging variational inference and a sparsely regularized Transformer architecture, the method substantially outperforms baseline approaches across four synthetic dynamical systems, successfully recovering highly disentangled representations and thereby validating the theoretical analysis.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Parametric system identification methods estimate the parameters of explicitly defined physical systems from data. Yet, they remain constrained by the need to provide an explicit function space, typically through a predefined library of candidate functions chosen via available domain knowledge. In contrast, deep learning can demonstrably model systems of broad complexity with high fidelity, but black-box function approximation typically fails to yield explicit descriptive or disentangled representations revealing the structure of a system. We develop a novel identifiability theorem, leveraging causal representation learning, to uncover disentangled representations of system parameters without structural assumptions. We derive a graphical criterion specifying when system parameters can be uniquely disentangled from raw trajectory data, up to permutation and diffeomorphism. Crucially, our analysis demonstrates that global causal structures provide a lower bound on the disentanglement guarantees achievable when considering local state-dependent causal structures. We instantiate system parameter identification as a variational inference problem, leveraging a sparsity-regularised transformer to uncover state-dependent causal structures. We empirically validate our approach across four synthetic domains, demonstrating its ability to recover highly disentangled representations that baselines fail to recover. Corroborating our theoretical analysis, our results confirm that enforcing local causal structure is often necessary for full identifiability.

Problem

• disentanglement
• dynamical systems
• causal representation learning
• system identification
• identifiability

Innovation

• causal representation learning
• disentangled representation
• system identification
• sparse attention
• identifiability