Sampling from Gaussian Processes: A Tutorial and Applications in Global Sensitivity Analysis and Optimization

2025-07-19

AI Summary

High-fidelity simulations and physical experiments are prohibitively expensive for global sensitivity analysis (GSA) and optimization. To address this, we propose an efficient uncertainty quantification framework based on Gaussian process (GP) surrogate models. Our method innovatively integrates random Fourier features (RFF) with pathwise conditional sampling to significantly enhance GP posterior sampling efficiency and scalability. We further extend the framework to single- and multi-objective Bayesian optimization and variance-based GSA. Extensive validation on numerical benchmarks and engineering-relevant test problems demonstrates that our approach achieves comparable or superior decision quality and sensitivity identification accuracy relative to conventional sampling strategies—while reducing computational overhead substantially. This work establishes a new paradigm for data-efficient analysis and optimization of high-cost engineering systems.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

High-fidelity simulations and physical experiments are essential for engineering analysis and design. However, their high cost often limits their applications in two critical tasks: global sensitivity analysis (GSA) and optimization. This limitation motivates the common use of Gaussian processes (GPs) as proxy regression models to provide uncertainty-aware predictions based on a limited number of high-quality observations. GPs naturally enable efficient sampling strategies that support informed decision-making under uncertainty by extracting information from a subset of possible functions for the model of interest. Despite their popularity in machine learning and statistics communities, sampling from GPs has received little attention in the community of engineering optimization. In this paper, we present the formulation and detailed implementation of two notable sampling methods -- random Fourier features and pathwise conditioning -- for generating posterior samples from GPs. Alternative approaches are briefly described. Importantly, we detail how the generated samples can be applied in GSA, single-objective optimization, and multi-objective optimization. We show successful applications of these sampling methods through a series of numerical examples.

Problem

• High cost limits simulations in sensitivity analysis and optimization.
• Gaussian Processes enable efficient sampling under uncertainty.
• Implement sampling methods for engineering optimization applications.

Innovation

• Uses Gaussian Processes for uncertainty-aware predictions
• Implements random Fourier features for sampling
• Applies pathwise conditioning in posterior sampling