Variational Neural Stochastic Differential Equations with Change Points

2024-11-01

arXiv.org

AI Summary

This work addresses time-series changepoint modeling under distributional shift. We propose a neural stochastic differential equation (SDE) framework built upon variational autoencoders. Our key contributions are threefold: (i) the first decoupling of Gaussian priors over initial states and sample paths in neural SDEs; (ii) two complementary, alternately optimizable changepoint estimation mechanisms; and (iii) a theoretical analysis of sequential likelihood ratio tests for changepoint detection. The method unifies variational inference, deep generative modeling, and maximum likelihood estimation. Empirical evaluation on both classical parametric SDE benchmarks and real-world distributionally shifted time-series datasets demonstrates significant improvements: higher changepoint localization accuracy, enhanced fidelity of synthetic time-series generation, and more calibrated uncertainty quantification.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In this work, we explore modeling change points in time-series data using neural stochastic differential equations (neural SDEs). We propose a novel model formulation and training procedure based on the variational autoencoder (VAE) framework for modeling time-series as a neural SDE. Unlike existing algorithms training neural SDEs as VAEs, our proposed algorithm only necessitates a Gaussian prior of the initial state of the latent stochastic process, rather than a Wiener process prior on the entire latent stochastic process. We develop two methodologies for modeling and estimating change points in time-series data with distribution shifts. Our iterative algorithm alternates between updating neural SDE parameters and updating the change points based on either a maximum likelihood-based approach or a change point detection algorithm using the sequential likelihood ratio test. We provide a theoretical analysis of this proposed change point detection scheme. Finally, we present an empirical evaluation that demonstrates the expressive power of our proposed model, showing that it can effectively model both classical parametric SDEs and some real datasets with distribution shifts.

Problem

• Modeling change points in time-series data using neural SDEs
• Developing a VAE-based training procedure for neural SDEs
• Detecting distribution shifts in time-series with change points

Innovation

• Variational neural SDEs for change points
• Gaussian prior on initial latent state
• Iterative likelihood-based change detection