Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data

2024-10-07

arXiv.org

AI Summary

Multi-source short time-series data suffer from limited per-sequence length, hindering accurate modeling of complex dynamical mechanisms. Method: We propose the first hierarchical unsupervised generative framework that jointly learns population-level shared priors and domain-specific dynamics. Our approach integrates variational inference, multi-domain dynamical system reconstruction (DSR), and interpretable latent-space learning to construct a linearly controllable, low-dimensional feature space—enabling cross-parameter-domain transfer and fundamental dynamical modeling. Contributions/Results: (1) First automatic discovery of interpretable dynamical features under a hierarchical structure; (2) High-fidelity single-domain reconstruction on standard DSR benchmarks and real-world neuroscience/clinical datasets; (3) Significantly improved generalization to unseen parameter regimes and modeling robustness in few-shot settings.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In science, we are often interested in obtaining a generative model of the underlying system dynamics from observed time series. While powerful methods for dynamical systems reconstruction (DSR) exist when data come from a single domain, how to best integrate data from multiple dynamical regimes and leverage it for generalization is still an open question. This becomes particularly important when individual time series are short, and group-level information may help to fill in for gaps in single-domain data. Here we introduce a hierarchical framework that enables to harvest group-level (multi-domain) information while retaining all single-domain characteristics, and showcase it on popular DSR benchmarks, as well as on neuroscience and medical data. In addition to faithful reconstruction of all individual dynamical regimes, our unsupervised methodology discovers common low-dimensional feature spaces in which datasets with similar dynamics cluster. The features spanning these spaces were further dynamically highly interpretable, surprisingly in often linear relation to control parameters that govern the dynamics of the underlying system. Finally, we illustrate transfer learning and generalization to new parameter regimes, paving the way toward DSR foundation models.

Problem

• Integrate data from multiple dynamical regimes
• Reconstruct individual and group-level dynamics
• Discover interpretable low-dimensional feature spaces

Innovation

• Hierarchical framework for multi-domain data integration
• Unsupervised discovery of low-dimensional feature spaces
• Interpretable linear relations to control parameters