Gradient-Free Sequential Bayesian Experimental Design via Interacting Particle Systems

2025-04-17

AI Summary

This work addresses the core challenge in sequential Bayesian optimal experimental design (BOED) for complex, gradient-free systems—such as high-dimensional inverse problems governed by PDEs—where nested expectations render the expected information gain (EIG) intractable and inefficient to estimate. We propose the first gradient-free, synergistic framework integrating ensemble Kalman inversion (EKI) with affine-invariant Langevin dynamics (ALDI), augmented by variational Gaussian approximation and parametric Laplace approximation to derive the first computationally tractable, tight upper and lower bounds on EIG. The entire method operates particle-based, requiring no gradients and imposing no smoothness assumptions on the forward model. Evaluated across diverse tasks—from linear Gaussian models to nonlinear PDE-constrained inverse problems—the approach significantly improves both EIG estimation efficiency and experimental design accuracy, thereby overcoming key scalability bottlenecks of BOED in nonsmooth, high-dimensional, and PDE-driven settings.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Bayesian Learning

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We introduce a gradient-free framework for Bayesian Optimal Experimental Design (BOED) in sequential settings, aimed at complex systems where gradient information is unavailable. Our method combines Ensemble Kalman Inversion (EKI) for design optimization with the Affine-Invariant Langevin Dynamics (ALDI) sampler for efficient posterior sampling-both of which are derivative-free and ensemble-based. To address the computational challenges posed by nested expectations in BOED, we propose variational Gaussian and parametrized Laplace approximations that provide tractable upper and lower bounds on the Expected Information Gain (EIG). These approximations enable scalable utility estimation in high-dimensional spaces and PDE-constrained inverse problems. We demonstrate the performance of our framework through numerical experiments ranging from linear Gaussian models to PDE-based inference tasks, highlighting the method's robustness, accuracy, and efficiency in information-driven experimental design.

Problem

• Develops gradient-free Bayesian Optimal Experimental Design for complex systems
• Addresses computational challenges in high-dimensional experimental design
• Proposes scalable utility estimation for PDE-constrained inverse problems

Innovation

• Gradient-free BOED framework for complex systems
• Combines EKI and ALDI for optimization and sampling
• Variational approximations for scalable utility estimation