🤖 AI Summary
In switchback experiments with unequal cluster sizes, between-unit variability inflates the variance of treatment effect estimators, thereby limiting statistical power. Conventional CUPAC methods fail to optimally reduce this variance because they do not distinguish between between-unit and within-unit noise. This work proposes a covariate adjustment approach explicitly designed to maximize statistical power by decomposing the variance structure of the treatment effect estimator and separately optimizing the trade-off between these two noise components during prediction modeling and residualization. We establish the first power-optimal CUPAC theoretical framework that jointly and distinctly accounts for different noise sources, moving beyond the traditional focus on overall prediction accuracy alone. Theoretically, the method achieves maximal statistical power; Monte Carlo simulations confirm its superior variance reduction over standard CUPAC and clarify the practical efficiency gains and applicability boundaries.
📝 Abstract
In switchback experiments with unequal cluster sizes, outcome dispersion across randomization units inflates estimator variance and limits statistical power. Standard control-using-prediction-as-covariate (CUPAC) adjustment may be suboptimal for variance reduction in this setting, because in its basic form it targets overall predictive accuracy and does not distinguish between the components of variance that vary across the randomization units of switchback experiments and those that vary across individual observations, even though these components contribute unequally to estimator variance. We propose a power-optimal variance reduction methodology via CUPAC that balances prediction of the noise between and within randomization units to achieve maximum statistical power. The methodology utilizes the framework for decomposition of the variance of the treatment-effect estimator for switchback experiments, and adapts both the outcome prediction and the analysis-time residualization to minimize the treatment-effect variance. The study first develops the theoretical framework for the power-optimal CUPAC. We then validate the theoretical framework through an extensive Monte Carlo simulation study. Finally, we discuss the practical considerations of the proposed methodology, including its potential efficiency gains and limitations.