Unbiased Regression-Adjusted Estimation of Average Treatment Effects in Randomized Controlled Trials

2025-11-05

AI Summary

In randomized controlled trials, conventional regression adjustment suffers from finite-sample bias under small samples, leading to unstable estimation and inaccurate inference. To address this, we propose LOORA (Leave-One-Out Regression Adjustment): it eliminates the finite-sample bias of regression-adjusted estimators via leave-one-out cross-validation—achieving unbiasedness in small samples for the first time; incorporates ridge regularization to enhance stability and precision; derives an exact finite-sample variance expression; and constructs a two-stage asymptotic variance estimator to ensure nominal coverage of confidence intervals. Theoretically, LOORA retains asymptotic efficiency. Empirically, it substantially reduces bias, achieves superior coverage probability, and demonstrates strong robustness and improved inferential performance under realistic potential outcome distributions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

This article introduces a leave-one-out regression adjustment estimator (LOORA) for estimating average treatment effects in randomized controlled trials. The method removes the finite-sample bias of conventional regression adjustment and provides exact variance expressions for LOORA versions of the Horvitz-Thompson and difference-in-means estimators under simple and complete random assignment. Ridge regularization limits the influence of high-leverage observations, improving stability and precision in small samples. In large samples, LOORA attains the asymptotic efficiency of regression-adjusted estimator as characterized by Lin (2013, Annals of Applied Statistics), while remaining exactly unbiased. To construct confidence intervals, we rely on asymptotic variance estimates that treat the estimator as a two-step procedure, accounting for both the regression adjustment and the random assignment stages. Two within-subject experimental applications that provide realistic joint distributions of potential outcomes as ground truth show that LOORA eliminates substantial biases and achieves close-to-nominal confidence interval coverage.

Problem

• Removes finite-sample bias in average treatment effect estimation
• Provides exact variance expressions for causal estimators under randomization
• Improves stability and precision in small samples via ridge regularization

Innovation

• Leave-one-out regression adjustment removes finite-sample bias
• Ridge regularization improves stability in small samples
• Asymptotic variance estimates enable accurate confidence intervals