Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Series

2024-10-08

arXiv.org

AI Summary

To address modeling challenges posed by irregular (non-uniformly sampled, sparse, discrete) time series prevalent in healthcare, climate science, and economics, this paper introduces a continuous-state-space dynamical modeling paradigm. Methodologically, it proposes: (1) the first multi-marginal Doob *h*-transform to construct interpretable conditional dynamical systems; (2) a tight evidence lower bound (ELBO) variational inference framework grounded in stochastic optimal control, enhancing posterior estimation accuracy; and (3) a hybrid architecture integrating simulation-free latent state evolution with a Transformer-based data assimilation module for efficient temporal awareness. Evaluated on diverse real-world irregular time series, the method achieves state-of-the-art performance across classification, regression, interpolation, and extrapolation tasks—demonstrating superior accuracy, strong generalization, and high training/inference efficiency compared to existing baselines.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Probabilistic Circuits and Graphical Models
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Many real-world datasets, such as healthcare, climate, and economics, are often collected as irregular time series, which poses challenges for accurate modeling. In this paper, we propose the Amortized Control of continuous State Space Model (ACSSM) for continuous dynamical modeling of time series for irregular and discrete observations. We first present a multi-marginal Doob's $h$-transform to construct a continuous dynamical system conditioned on these irregular observations. Following this, we introduce a variational inference algorithm with a tight evidence lower bound (ELBO), leveraging stochastic optimal control (SOC) theory to approximate the intractable Doob's $h$-transform and simulate the conditioned dynamics. To improve efficiency and scalability during both training and inference, ACSSM employs amortized inference to decouple representation learning from the latent dynamics. Additionally, it incorporates a simulation-free latent dynamics framework and a transformer-based data assimilation scheme, facilitating parallel inference of the latent states and ELBO computation. Through empirical evaluations across a variety of real-world datasets, ACSSM demonstrates superior performance in tasks such as classification, regression, interpolation, and extrapolation, while maintaining computational efficiency.

Problem

• Model irregular time series data
• Amortized Control for dynamical modeling
• Efficient and scalable inference methods

Innovation

• Amortized Control State Space Model
• Variational Inference with ELBO
• Transformer-based Data Assimilation