Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design

2025-07-18

AI Summary

Fully amortized Bayesian experimental design (BED) lacks online adaptability, while non-amortized methods incur prohibitive computational costs. Method: We propose a semi-amortized, stepwise deep adaptive design framework featuring test-time periodic policy updates—dynamically refining a neural policy network based on current observations during experimentation. This balances the efficiency of full amortization with the instance-specificity of non-amortized approaches. Our method jointly leverages Bayesian inference and policy-gradient reinforcement learning, enabling end-to-end differentiable training and online decision-making. Results: Evaluated across diverse benchmarks—including PCR kinetic modeling and active learning—our approach consistently outperforms state-of-the-art methods, achieving an average 12.7% improvement in decision utility. Moreover, it demonstrates superior robustness to model misspecification and observational noise.

Technology Category

• Machine Learning: Bayesian Learning
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We develop a semi-amortized, policy-based, approach to Bayesian experimental design (BED) called Stepwise Deep Adaptive Design (Step-DAD). Like existing, fully amortized, policy-based BED approaches, Step-DAD trains a design policy upfront before the experiment. However, rather than keeping this policy fixed, Step-DAD periodically updates it as data is gathered, refining it to the particular experimental instance. This test-time adaptation improves both the flexibility and the robustness of the design strategy compared with existing approaches. Empirically, Step-DAD consistently demonstrates superior decision-making and robustness compared with current state-of-the-art BED methods.

Problem

• Develops adaptive Bayesian experimental design method
• Updates policy during data collection for flexibility
• Improves decision-making robustness in experiments

Innovation

• Semi-amortized policy-based Bayesian experimental design
• Periodic policy updates during data gathering
• Improved flexibility and robustness in design