Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators

2025-11-24

AI Summary

This paper addresses the adaptive Neyman allocation problem for the augmented inverse probability weighting (AIPW) estimator during the design phase: dynamically selecting treatment assignment probabilities and linear predictors as subjects arrive sequentially, to minimize the Neyman regret—the gap between the adaptive variance and the oracle-optimal non-adaptive variance. We propose Sigmoid-FTRL, the first algorithm that decomposes non-convex regret minimization into two jointly optimized convex subproblems, achieving the optimal $T^{-1/2}R^2$ convergence rate—proven unimprovable under standard regularity conditions. Leveraging online convex optimization, the Follow-the-Regularized-Leader (FTRL) framework, and conservative variance estimation, we establish asymptotic normality of the AIPW estimator and construct asymptotically valid Wald-type confidence intervals. These contributions substantially enhance statistical precision and reliability in adaptive causal inference.

Technology Category

• Machine Learning: Causal Learning
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We consider the problem of Adaptive Neyman Allocation for the class of AIPW estimators in a design-based setting, where potential outcomes and covariates are deterministic. As each subject arrives, an adaptive procedure must select both a treatment assignment probability and a linear predictor to be used in the AIPW estimator. Our goal is to construct an adaptive procedure that minimizes the Neyman Regret, which is the difference between the variance of the adaptive procedure and an oracle variance which uses the optimal non-adaptive choice of assignment probability and linear predictors. While previous work has drawn insightful connections between Neyman Regret and online convex optimization for the Horvitz--Thompson estimator, one of the central challenges for AIPW estimator is that the underlying optimization is non-convex. In this paper, we propose Sigmoid-FTRL, an adaptive experimental design which addresses the non-convexity via simultaneous minimization of two convex regrets. We prove that under standard regularity conditions, the Neyman Regret of Sigmoid-FTRL converges at a $T^{-1/2} R^2$ rate, where $T$ is the number of subjects in the experiment and $R$ is the maximum norm of covariate vectors. Moreover, we show that no adaptive design can improve upon the $T^{-1/2}$ rate under our regularity conditions. Finally, we establish a central limit theorem and a consistently conservative variance estimator which facilitate the construction of asymptotically valid Wald-type confidence intervals.

Problem

• Minimizing Neyman Regret for AIPW estimators in adaptive experiments
• Addressing non-convex optimization via simultaneous convex regret minimization
• Achieving optimal T^{-1/2} convergence rate under regularity conditions

Innovation

• Addresses non-convexity via dual convex regrets
• Proposes Sigmoid-FTRL adaptive experimental design
• Achieves T^{-1/2} rate for Neyman Regret minimization