Uncertainty quantification of neural network models of evolving processes via Langevin sampling

2025-04-21

AI Summary

This work addresses uncertainty modeling for history-dependent dynamical processes—such as chemical reactions and material evolution—by proposing a joint Bayesian data model integrating neural ordinary differential equations (NODEs) with a trainable observation model. Methodologically, it introduces the first framework embedding Langevin dynamics sampling within a hypernetwork architecture, enabling end-to-end joint learning of data model parameters and the posterior score function, thereby balancing computational efficiency and posterior approximation accuracy. Leveraging score-matching-driven Bayesian approximate inference, the approach circumvents restrictive mean-field assumptions. Experiments on diverse physicochemical datasets demonstrate substantial improvements over mean-field variational inference, enabling high-confidence quantification of parameter uncertainties and efficient posterior sampling.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We propose a scalable, approximate inference hypernetwork framework for a general model of history-dependent processes. The flexible data model is based on a neural ordinary differential equation (NODE) representing the evolution of internal states together with a trainable observation model subcomponent. The posterior distribution corresponding to the data model parameters (weights and biases) follows a stochastic differential equation with a drift term related to the score of the posterior that is learned jointly with the data model parameters. This Langevin sampling approach offers flexibility in balancing the computational budget between the evaluation cost of the data model and the approximation of the posterior density of its parameters. We demonstrate performance of the hypernetwork on chemical reaction and material physics data and compare it to mean-field variational inference.

Problem

• Quantify uncertainty in neural network models of evolving processes
• Develop scalable inference for history-dependent processes via hypernetworks
• Balance computational cost between model evaluation and posterior approximation

Innovation

• Scalable hypernetwork framework for history-dependent processes
• Neural ODE with trainable observation model
• Langevin sampling for posterior distribution approximation