Stronger Neyman Regret Guarantees for Adaptive Experimental Design

2025-02-24

AI Summary

This paper addresses the design of adaptive sequential experiments for unbiased estimation of the average treatment effect (ATE), aiming to match the efficiency of the hindsight-optimal non-adaptive design while achieving sublinear Neyman regret. Method: We propose an improved ClipOGD algorithm that integrates covariate-driven, multi-group weighted online optimization with an adaptive assignment mechanism. Contribution/Results: We present the first anytime algorithm achieving $ ilde{O}(log T)$ Neyman regret for single-group settings, and extend it to contextual multi-group settings—introducing a novel paradigm of group-wise adaptive optimality under covariate grouping. Theoretically, our method attains anytime $ ilde{O}(log T)$ regret in the single-group case and $ ilde{O}(sqrt{T})$ in the multi-group case. Empirical results demonstrate substantial improvements over state-of-the-art baselines in multi-group scenarios.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We study the design of adaptive, sequential experiments for unbiased average treatment effect (ATE) estimation in the design-based potential outcomes setting. Our goal is to develop adaptive designs offering sublinear Neyman regret, meaning their efficiency must approach that of the hindsight-optimal nonadaptive design. Recent work [Dai et al, 2023] introduced ClipOGD, the first method achieving $widetilde{O}(sqrt{T})$ expected Neyman regret under mild conditions. In this work, we propose adaptive designs with substantially stronger Neyman regret guarantees. In particular, we modify ClipOGD to obtain anytime $widetilde{O}(log T)$ Neyman regret under natural boundedness assumptions. Further, in the setting where experimental units have pre-treatment covariates, we introduce and study a class of contextual"multigroup"Neyman regret guarantees: Given any set of possibly overlapping groups based on the covariates, the adaptive design outperforms each group's best non-adaptive designs. In particular, we develop a contextual adaptive design with $widetilde{O}(sqrt{T})$ anytime multigroup Neyman regret. We empirically validate the proposed designs through an array of experiments.

Problem

• adaptive experimental design
• Neyman regret guarantees
• unbiased ATE estimation

Innovation

• Modified ClipOGD algorithm
• Anytime logarithmic Neyman regret
• Contextual multigroup adaptive design