MD-NOMAD: Mixture density nonlinear manifold decoder for emulating stochastic differential equations and uncertainty propagation

2024-04-24

arXiv.org

AI Summary

Modeling stochastic differential equations (SDEs) and propagating uncertainty through complex dynamical systems remains challenging, particularly due to the limitations of deterministic neural operators in capturing non-Gaussian, high-dimensional stochastic outputs. Method: We propose MD-NOMAD—a novel framework that couples pointwise neural operators with mixture density estimation (realized via Gaussian Mixture Networks, MDNs) and a nonlinear manifold decoder (NOMAD) to model the conditional probability distribution of stochastic output functions. Contribution/Results: MD-NOMAD overcomes the expressivity constraints of conventional deterministic neural operators while preserving scalability to high-dimensional settings. It accurately characterizes intricate, non-Gaussian uncertainty structures and achieves state-of-the-art performance across diverse SDE and stochastic PDE surrogate modeling tasks—faithfully reproducing uncertainty propagation behavior. Empirically, it improves training efficiency by over 40%, exhibits strong generalization and robustness, and establishes a scalable, probabilistically consistent paradigm for uncertainty quantification in stochastic systems.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We propose a neural operator framework, termed mixture density nonlinear manifold decoder (MD-NOMAD), for stochastic simulators. Our approach leverages an amalgamation of the pointwise operator learning neural architecture nonlinear manifold decoder (NOMAD) with mixture density-based methods to estimate conditional probability distributions for stochastic output functions. MD-NOMAD harnesses the ability of probabilistic mixture models to estimate complex probability and the high-dimensional scalability of pointwise neural operator NOMAD. We conduct empirical assessments on a wide array of stochastic ordinary and partial differential equations and present the corresponding results, which highlight the performance of the proposed framework.

Problem

• Emulating stochastic differential equations with neural operators
• Estimating conditional probability distributions for stochastic outputs
• Scaling probabilistic models for high-dimensional uncertainty propagation

Innovation

• Neural operator framework MD-NOMAD for stochastic simulators
• Combines NOMAD with mixture density methods
• Estimates complex probability distributions scalably