Semiparametric Efficiency in Sequential Experiments: Characterization and Design via Average Propensity

2026-06-30

AI Summary

This study addresses causal effect estimation in non-i.i.d. sequential experiments, such as AI service evaluations, by establishing the average propensity score as a universal efficiency benchmark for the first time. It reframes efficient experimental design as the problem of learning an optimal allocation rule and proposes two complementary approaches: regression adjustment based on the efficient influence function and an adaptive covariate balancing mechanism. These are integrated with operational constraints to yield a practical batch-adaptive design. Under standard perturbation convergence conditions, the proposed method achieves the semiparametric efficiency bound for linear functionals, attaining sharp second-order convergence rates. Numerical experiments and an evaluation of an AI-powered medical assistant demonstrate that this framework substantially improves estimation efficiency in multi-treatment settings.

Abstract

Modern experiments, including evaluations of AI-enabled services and platform interventions, often depart from independent and identically distributed (i.i.d.) sampling because assignments may be adaptive, balanced across covariates, or subject to rollout constraints such as exposure, fairness, and budget limits. This paper studies the efficiency benchmark for estimating causal targets in such sequential experiments. We show that every non-anticipating design induces an average propensity score, and we establish a semiparametric lower bound: for regular locally unbiased estimators, attainable precision is bounded by the i.i.d. efficiency benchmark evaluated at this induced score. The average propensity score thereby serves as a common benchmark and design target, allowing sequential experimental design to be viewed as choosing or learning an efficient allocation rule, with operational constraints entering through the admissible set when present. We then develop implementable batched adaptive designs that approach this benchmark through two complementary mechanisms. The first uses regression adjustment based on efficient influence functions; for general smooth estimands it attains the benchmark under standard nuisance-rate conditions, while for linear functionals of outcome means it achieves a sharp second-order rate. The second uses adaptive covariate balancing to attain the same benchmark through the assignment mechanism, enabling simple moment-based estimation. Both routes require only a small number of policy updates, making them compatible with delayed feedback and easier to monitor in operational deployments. Numerical experiments and an empirical study of AI medical-assistant evaluation demonstrate the practical efficiency gains, including in multi-treatment settings. Overall, the paper provides a unified framework for characterizing and designing efficient sequential experiments.

Problem

• sequential experiments
• causal inference
• semiparametric efficiency
• average propensity score
• non-i.i.d. sampling

Innovation

• semiparametric efficiency
• sequential experiments
• average propensity score
• adaptive design
• covariate balancing