Bayesian Design of Experiments in the Presence of Nuisance Parameters

2025-08-05

AI Summary

Bayesian experimental design with nuisance parameters remains challenging due to the need to account for their prior uncertainty while maintaining statistical rigor and computational feasibility. Method: This paper proposes a fully Bayesian framework that explicitly models prior uncertainty over all parameters—including nuisance parameters—during the design stage. It introduces Bayesian additive regression trees (BART) to the experimental design literature for the first time, integrating asymptotic posterior approximations with Monte Carlo simulation to efficiently optimize sample size and decision rules under both fixed and adaptive designs. Contribution/Results: The approach significantly reduces computational burden compared to conventional resampling-intensive methods, while preserving statistical power and robust operating characteristics. Key innovations include: (1) unified quantification of nuisance parameter uncertainty; (2) a BART-driven design function learning mechanism that enhances interpretability and generalizability; and (3) an end-to-end Bayesian design pipeline balancing robustness, efficiency, and practical implementation.

Technology Category

• Machine Learning: Bayesian Learning
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Design of experiments has traditionally relied on the frequentist hypothesis testing framework where the optimal size of the experiment is specified as the minimum sample size that guarantees a required level of power. Sample size determination may be performed analytically when the test statistic has a known asymptotic sampling distribution and, therefore, the power function is available in analytic form. Bayesian methods have gained popularity in all stages of discovery, namely, design, analysis and decision making. Bayesian decision procedures rely on posterior summaries whose sampling distributions are commonly estimated via Monte Carlo simulations. In the design of scientific studies, the Bayesian approach incorporates uncertainty about the design value(s) instead of conditioning on a single value of the model parameter(s). Accounting for uncertainties in the design value(s) is particularly critical when the model includes nuisance parameters. In this manuscript, we propose methodology that utilizes the large-sample properties of the posterior distribution together with Bayesian additive regression trees (BART) to efficiently obtain the optimal sample size and decision criteria in fixed and adaptive designs. We introduce a fully Bayesian procedure that incorporates the uncertainty associated with the model parameters including the nuisance parameters at the design stage. The proposed approach significantly reduces the computational burden associated with Bayesian design and enables the wide adoption of Bayesian operating characteristics.

Problem

• Optimize sample size in experiments with nuisance parameters
• Develop Bayesian method for efficient experimental design
• Reduce computational cost in Bayesian design procedures

Innovation

• Uses Bayesian additive regression trees (BART)
• Incorporates uncertainty in model parameters
• Reduces computational burden in Bayesian design