Sequentially-Rerandomized Switchback Experiments

2026-04-02

AI Summary

This study addresses the challenges posed by sparse experimental units, strong heterogeneity, non-stationarity, and cross-period spillover effects in large-scale online platforms, which severely undermine the efficiency and induce substantial estimation bias in conventional A/B tests. To overcome these limitations, the authors propose a Sequential Re-randomization Switchback (SRSB) design that dynamically constructs prognostic covariates from historical observations at each time period to enhance balance between treatment groups and improve estimation precision. The approach innovatively incorporates time-dependent information and is extended to first-order spillover settings via a block-wise SRSB formulation that establishes stable “persistent” treatment groups. Leveraging both finite-sample randomization inference and asymptotic methods, the framework accommodates diverse experimental scenarios with or without spillovers. Simulations demonstrate that SRSB substantially outperforms standard switchback designs in both accuracy and robustness.

Abstract

Large-scale online platforms and marketplace systems often evaluate new policies through experiments that randomize treatment across operational units (e.g., geographies, regions, or clusters) over many time periods. In these settings, standard A/B testing can be inefficient or unreliable due to a limited number of units, substantial cross-unit heterogeneity, non-stationarity, and potential carryover across periods. We propose Sequentially-Rerandomized Switchback Experiments (SRSB), a new experimental design that helps mitigate these challenges. SRSB re-randomizes treatment at each time period such as to enforce balance on pre-specified prognostic variables constructed from past observations. In the absence of carryover, SRSB improves precision by leveraging temporal dependence through balancing lagged outcomes and covariates; we develop finite-sample randomization inference under a sharp null as well as asymptotic inference as the number of periods grows. We then extend SRSB to settings with first-order carryover and introduce a blocked SRSB variant that rerandomizes within strata defined by the previous treatment to form stable and comparable "stay" groups. Extensive simulations demonstrate the practical gains and robustness of SRSB relative to standard switchback designs.

Problem

• switchback experiments
• carryover effects
• cross-unit heterogeneity
• non-stationarity
• treatment randomization

Innovation

• Sequentially-Rerandomized Switchback
• carryover effects
• randomization inference
• temporal dependence
• experimental design