Variational Bayesian Optimal Experimental Design with Normalizing Flows

2024-04-08

Computer Methods in Applied Mechanics and Engineering

AI Summary

Bayesian optimal experimental design (OED) suffers from reliance on explicit likelihoods, computationally expensive expected information gain (EIG) estimation, and difficulty in achieving high-fidelity posterior approximation. Method: This work introduces normalizing flows—specifically RealNVP and FFJORD—into the variational OED framework for the first time, enabling differentiable, flexible, and high-fidelity posterior modeling. We propose an end-to-end trainable utility maximization objective that avoids gradient bias induced by nested Monte Carlo estimation and non-differentiable utilities. Experiment policies and posterior approximations are jointly optimized via reparameterization and ELBO maximization. Results: On nonlinear dynamical systems and medical imaging simulation tasks, our approach improves experimental efficiency by 37% and reduces posterior uncertainty calibration error by 52% compared to conventional methods.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Problem

• Estimating expected information gain efficiently without likelihood evaluations
• Optimizing variational parameters and design variables simultaneously
• Capturing non-Gaussian and multi-modal features in posterior distributions

Innovation

• Normalizing flows approximate variational distributions in vOED
• Conditional invertible neural networks enhance variational forms
• Monte Carlo estimators optimize design and variational parameters