SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations

2025-02-04

AI Summary

To address the scalability bottleneck in latent stochastic differential equation (SDE) training—stemming from reliance on numerical simulation and adjoint-based sensitivity backpropagation—this paper introduces SDE Matching, a novel training paradigm. It is the first to adapt Score/Flow Matching principles to stochastic dynamical modeling, enabling fully simulation-free learning. Our method formulates distribution matching over stochastic trajectories in the latent space, parameterizes the dynamics via neural SDEs, and employs implicit gradient estimation to bypass explicit SDE solving and adjoint integration. Experiments on multiple time-series benchmarks demonstrate that SDE Matching achieves modeling accuracy comparable to the adjoint method while accelerating training by 3–5× and substantially reducing memory consumption. Moreover, it natively supports long sequences and high-dimensional latent representations, overcoming key limitations of conventional approaches.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The Latent Stochastic Differential Equation (SDE) is a powerful tool for time series and sequence modeling. However, training Latent SDEs typically relies on adjoint sensitivity methods, which depend on simulation and backpropagation through approximate SDE solutions, which limit scalability. In this work, we propose SDE Matching, a new simulation-free method for training Latent SDEs. Inspired by modern Score- and Flow Matching algorithms for learning generative dynamics, we extend these ideas to the domain of stochastic dynamics for time series and sequence modeling, eliminating the need for costly numerical simulations. Our results demonstrate that SDE Matching achieves performance comparable to adjoint sensitivity methods while drastically reducing computational complexity.

Problem

• training Latent SDEs
• eliminating simulation reliance
• reducing computational complexity

Innovation

• Simulation-free Latent SDE training
• Score- and Flow Matching inspired
• Reduced computational complexity significantly