Does Rerandomization Help Beyond Covariate Adjustment? A Review and Guide for Theory and Practice

2025-12-04

AI Summary

While covariate-adjusted estimators (e.g., linear regression, matching) are widely used in causal inference, it remains unclear whether rerandomization—despite such adjustments—still delivers meaningful benefits, particularly in finite samples where existing asymptotic theory fails to capture non-precision advantages (e.g., estimator consistency) or practical performance. Method: We conduct large-scale simulation studies to systematically evaluate rerandomization’s impact on estimation precision, statistical power, confidence interval coverage, and consistency across multiple estimators, complemented by theoretical analysis. Contribution/Results: Rerandomization substantially improves finite-sample estimation accuracy, robustness, and consistency of causal effect estimates; enhances statistical power; and reduces false-positive rates—even when covariate adjustment is already employed. These gains extend beyond asymptotic guarantees, offering practitioners a principled, efficiency-enhancing, and reliability-improving design strategy for randomized experiments.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements

Abstract

Rerandomization is a modern experimental design technique that repeatedly randomizes treatment assignments until covariates are deemed balanced between treatment groups. This enhances the precision and coherence of causal effect estimators, mitigates false discoveries from p-hacking, and increases statistical power. Recent work suggests that balancing covariates via rerandomization does not alter the asymptotic precision of covariate-adjusted estimators, thereby making it unclear whether rerandomization is worthwhile if adjusted estimators are used. However, these results have two key caveats. First, these results are asymptotic, leaving finite sample performance unknown. Second, these results focus on precision, while other potential benefits, such as increased coherence among flexible estimators, remain understudied. Hence, in this paper we provide three main contributions: (i) a comprehensive review of the rerandomization literature, covering historical foundations, theoretical developments, and recent methodological advancements, (ii) an extensive simulation study examining finite-sample performance, and (iii) a practical guide for practitioners. Our study compares precision, coherence, power, and coverage of various estimators under rerandomization versus complete randomization. We find rerandomization to be a complementary design strategy that enhances the precision, robustness, and reliability of causal effect estimators, especially for smaller sample sizes.

Problem

• Evaluates rerandomization's benefits beyond covariate adjustment in experiments
• Examines finite-sample performance and estimator coherence under rerandomization
• Compares precision, power, and coverage between rerandomization and complete randomization

Innovation

• Rerandomization balances covariates through repeated treatment assignment randomization
• Simulation study evaluates finite-sample performance of rerandomization versus complete randomization
• Rerandomization enhances precision, robustness, and reliability of causal estimators