Optimal Data Integration and Adaptive Sampling for Efficient Treatment Effect Estimation

2026-03-30

AI Summary

This study addresses the challenge of estimating the average treatment effect (ATE) of advertising campaigns in online marketplaces, where fully randomized experiments are often infeasible due to practical and cost constraints, leading to biased estimates. To overcome this, the authors propose a Bayesian adaptive experimental design framework that integrates observational and experimental data. Central to their approach is an optimal shrinkage estimator that achieves accurate ATE estimation without requiring smoothness assumptions on treatment effects. The method effectively balances bias-variance trade-offs while substantially reducing the required scale of randomization. Theoretical guarantees include asymptotic normality and regret bounds. Empirical evaluation on 2,583 advertising campaigns from Amazon Ads demonstrates that the proposed approach attains the same estimation accuracy as conventional methods using only half the randomized sample size.

Abstract

This study addresses the challenge of estimating average treatment effects (ATEs) for advertising campaigns in online marketplaces where complete randomized experimentation is infeasible. We propose two key innovations: (1) a shrinkage estimator that optimally combines observational and experimental data without assuming smooth treatment effects across campaigns, and (2) a Bayesian adaptive experimental design framework that efficiently selects campaigns for randomized evaluation that minimizes cumulative risk. Our shrinkage estimator achieves lower risk compared to existing methods by balancing bias-variance tradeoffs, while our adaptive design significantly reduces the costs of campaign randomization. We establish theoretical guarantees including asymptotic normality and regret bounds. In an application to Amazon Ads data analyzing 2,583 campaigns, our approach achieves equivalent estimation precision while requiring only half of the randomized experiments needed by random sampling, the standard method widely used in practice today. The proposed method serves as a practical solution for marketplace platforms to efficiently measure advertising effectiveness while managing experimentation costs.

Problem

• average treatment effect
• online marketplaces
• randomized experimentation
• advertising campaigns
• data integration

Innovation

• shrinkage estimator
• Bayesian adaptive design
• treatment effect estimation
• data integration
• experimental efficiency