FMint-SDE: A Multimodal Foundation Model for Accelerating Numerical Simulation of SDEs via Error Correction

2025-10-31

AI Summary

Numerical solvers for stochastic differential equations (SDEs) face a fundamental trade-off between accuracy and computational efficiency, while existing neural solvers suffer from poor generalizability and require system-specific retraining. Method: We propose the first initialization-based multimodal foundation model specifically designed for SDEs. Built upon a decoder-only Transformer architecture, it jointly models numerical solution trajectories and textual prompts, enabling in-context error correction of coarse approximations from classical solvers. Contribution/Results: Our model achieves cross-system generalization and zero-shot or few-shot transfer without retraining on new dynamical systems. Evaluated across benchmarks in molecular dynamics, mechanical systems, finance, and biology, it outperforms classical methods—including Euler–Maruyama and Milstein—by 1–2 orders of magnitude in accuracy at comparable computational cost, or reduces runtime by over 50% at equivalent accuracy, thereby establishing a superior accuracy–efficiency Pareto frontier.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Fast and accurate simulation of dynamical systems is a fundamental challenge across scientific and engineering domains. Traditional numerical integrators often face a trade-off between accuracy and computational efficiency, while existing neural network-based approaches typically require training a separate model for each case. To overcome these limitations, we introduce a novel multi-modal foundation model for large-scale simulations of differential equations: FMint-SDE (Foundation Model based on Initialization for stochastic differential equations). Based on a decoder-only transformer with in-context learning, FMint-SDE leverages numerical and textual modalities to learn a universal error-correction scheme. It is trained using prompted sequences of coarse solutions generated by conventional solvers, enabling broad generalization across diverse systems. We evaluate our models on a suite of challenging SDE benchmarks spanning applications in molecular dynamics, mechanical systems, finance, and biology. Experimental results show that our approach achieves a superior accuracy-efficiency tradeoff compared to classical solvers, underscoring the potential of FMint-SDE as a general-purpose simulation tool for dynamical systems.

Problem

• Accelerating SDE simulations via multimodal error correction
• Overcoming accuracy-efficiency tradeoff in numerical integrators
• Developing universal solver eliminating per-case model training

Innovation

• Multimodal foundation model corrects numerical simulation errors
• Transformer learns universal error-correction from coarse solutions
• Model generalizes across diverse stochastic differential equation systems