Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs

2025-07-02

AI Summary

This work addresses numerical integration of high-dimensional probability distributions. We propose a novel integration method combining Neural Ordinary Differential Equation (NeuralODE)-based transport maps with Clenshaw–Curtis sparse grids. Our key contributions are threefold: (i) We establish, for the first time, a Probably Approximately Correct (PAC) learning-theoretic guarantee on the error controllability of NeuralODE-driven sparse grid integration within an empirical risk minimization framework; (ii) we jointly analyze statistical and numerical integration errors, enabling unified modeling and consistent theoretical justification; (iii) we prove that, under conditions where sample size grows and neural network capacity scales adaptively, the integral estimator converges to the true value with high probability and arbitrary accuracy. This framework bridges theoretical rigor and computational feasibility, offering a principled approach for high-dimensional Bayesian inference and related probabilistic computing tasks.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper provides a proof of the consistency of sparse grid quadrature for numerical integration of high dimensional distributions. In a first step, a transport map is learned that normalizes the distribution to a noise distribution on the unit cube. This step is built on the statistical learning theory of neural ordinary differential equations, which has been established recently. Secondly, the composition of the generative map with the quantity of interest is integrated numerically using the Clenshaw-Curtis sparse grid quadrature. A decomposition of the total numerical error in quadrature error and statistical error is provided. As main result it is proven in the framework of empirical risk minimization that all error terms can be controlled in the sense of PAC (probably approximately correct) learning and with high probability the numerical integral approximates the theoretical value up to an arbitrary small error in the limit where the data set size is growing and the network capacity is increased adaptively.

Problem

• Proves consistency of sparse grid quadrature for high-dimensional integration
• Learns transport map to normalize distributions using NeuralODEs
• Controls quadrature and statistical errors via PAC learning framework

Innovation

• NeuralODEs learn transport map for normalization
• Clenshaw-Curtis sparse grid quadrature integration
• PAC learning controls quadrature and statistical errors