Bayesian Quadrature: Gaussian Processes for Integration

2026-02-18

AI Summary

This study addresses the challenge of evaluating intractable integrals or expectations by providing a systematic review of Bayesian quadrature methods. It establishes, for the first time, a unified taxonomic framework encompassing three core dimensions: modeling, inference, and sampling. Building upon Gaussian process–based probabilistic modeling and integrating Bayesian inference with numerical integration techniques, the paper elucidates the underlying mathematical foundations, synthesizes interdisciplinary literature, and assesses—through controlled numerical experiments—the impact of various design choices on empirical performance. Beyond offering a comprehensive theoretical overview and an extensive bibliography, this work critically examines practical challenges and limitations inherent in current approaches, thereby laying a cohesive foundation for future research in the field.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Probabilistic Inference

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Bayesian quadrature is a probabilistic, model-based approach to numerical integration, the estimation of intractable integrals, or expectations. Although Bayesian quadrature was popularised already in the 1980s, no systematic and comprehensive treatment has been published. The purpose of this survey is to fill this gap. We review the mathematical foundations of Bayesian quadrature from different points of view; present a systematic taxonomy for classifying different Bayesian quadrature methods along the three axes of modelling, inference, and sampling; collect general theoretical guarantees; and provide a controlled numerical study that explores and illustrates the effect of different choices along the axes of the taxonomy. We also provide a realistic assessment of practical challenges and limitations to application of Bayesian quadrature methods and include an up-to-date and nearly exhaustive bibliography that covers not only machine learning and statistics literature but all areas of mathematics and engineering in which Bayesian quadrature or equivalent methods have seen use.

Problem

• Bayesian quadrature
• numerical integration
• probabilistic numerical methods
• Gaussian processes
• systematic review

Innovation

• Bayesian quadrature
• Gaussian processes
• probabilistic numerical integration
• systematic taxonomy
• controlled numerical study