Sequential Design with Posterior and Posterior Predictive Probabilities

2025-04-01

AI Summary

In Bayesian sequential trials, error rate evaluation relies on computationally expensive Monte Carlo simulations, hindering efficient optimization of sample size and decision thresholds. Method: This paper establishes, for the first time, analytical functional relationships between posterior and posterior predictive probabilities and sample size. Leveraging Bayesian decision theory and asymptotic analysis—combined with numerical fitting and error-rate inversion—the method enables precise error-rate assessment for any sample size using only two simulations, and rapidly identifies optimal design parameters. Contribution/Results: The approach drastically reduces computational cost while achieving error-rate control accuracy comparable to conventional simulation-based methods. In two real-world case studies, it attains exact error-rate calibration and accelerates design optimization by several orders of magnitude. This provides a scalable, verifiable, and highly efficient design paradigm for Bayesian adaptive trials.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Sequential designs drive innovation in clinical, industrial, and corporate settings. Early stopping for failure in sequential designs conserves experimental resources, whereas early stopping for success accelerates access to improved interventions. Bayesian decision procedures provide a formal and intuitive framework for early stopping using posterior and posterior predictive probabilities. Design parameters including decision thresholds and sample sizes are chosen to control the error rates associated with the sequential decision process. These choices are routinely made based on estimating the sampling distribution of posterior summaries via intensive Monte Carlo simulation for each sample size and design scenario considered. In this paper, we propose an efficient method to assess error rates and determine optimal sample sizes and decision thresholds for Bayesian sequential designs. We prove theoretical results that enable posterior and posterior predictive probabilities to be modeled as a function of the sample size. Using these functions, we assess error rates at a range of sample sizes given simulations conducted at only two sample sizes. The effectiveness of our methodology is highlighted using two substantive examples.

Problem

• Efficient error rate assessment for Bayesian sequential designs
• Optimal sample size determination using posterior probabilities
• Modeling probabilities as functions of sample size

Innovation

• Bayesian decision procedures for early stopping
• Modeling probabilities as sample size functions
• Error rate assessment with minimal simulations