Multiple Randomization Designs

📅 2021-12-27
🏛️ arXiv.org
📈 Citations: 48
Influential: 3
📄 PDF
🤖 AI Summary
Traditional randomized controlled trials (RCTs) suffer from cross-group spillovers and interference in multi-population interaction settings—e.g., buyer-seller or creator-subscriber systems—leading to biased estimation of the average treatment effect (ATE). This paper introduces the first systematic multidimensional randomization design framework, relaxing the conventional single-layer randomization assumption. By integrating hierarchical randomization, cross-group assignment, and potential outcomes modeling, it jointly identifies both the ATE and cross-group interference effects. We establish theoretical guarantees: the proposed unbiased estimator is consistent and asymptotically normal. Simulation results demonstrate substantially higher statistical power compared to standard RCTs. This work extends the scope of causal questions addressable through experimental design and provides a rigorous foundation for causal inference in complex intervention environments—particularly platform economies—where interdependent user behaviors induce non-negligible interference.
📝 Abstract
In this study we introduce a new class of experimental designs. In a classical randomized controlled trial (RCT), or A/B test, a randomly selected subset of a population of units (e.g., individuals, plots of land, or experiences) is assigned to a treatment (treatment A), and the remainder of the population is assigned to the control treatment (treatment B). The difference in average outcome by treatment group is an estimate of the average effect of the treatment. However, motivating our study, the setting for modern experiments is often different, with the outcomes and treatment assignments indexed by multiple populations. For example, outcomes may be indexed by buyers and sellers, by content creators and subscribers, by drivers and riders, or by travelers and airlines and travel agents, with treatments potentially varying across these indices. Spillovers or interference can arise from interactions between units across populations. For example, sellers' behavior may depend on buyers' treatment assignment, or vice versa. This can invalidate the simple comparison of means as an estimator for the average effect of the treatment in classical RCTs. We propose new experiment designs for settings in which multiple populations interact. We show how these designs allow us to study questions about interference that cannot be answered by classical randomized experiments. Finally, we develop new statistical methods for analyzing these Multiple Randomization Designs.
Problem

Research questions and friction points this paper is trying to address.

Addressing interference effects in multi-population experimental settings
Proposing new designs for experiments with interacting populations
Developing statistical methods for analyzing multiple randomization designs
Innovation

Methods, ideas, or system contributions that make the work stand out.

Multiple Randomization Designs for interacting populations
Addresses interference issues in classical RCTs
New statistical methods for analyzing these designs
🔎 Similar Papers
No similar papers found.
Lorenzo Masoero
Lorenzo Masoero
Applied Scientist, Amazon
Bayesian inferenceCausal InferenceReplicability
Suhas Vijaykumar
Suhas Vijaykumar
Amazon Science post-doc (Amazon.com)
EconometricsMachine LearningStatistics
Thomas S. Richardson
Thomas S. Richardson
University of Washington
CausalityCounterfactualsGraphical Models
J
James McQueen
Amazon, US
Ido Rosen
Ido Rosen
Amazon, US
B
Brian Burdick
Work done while at Amazon, US
P
Pat Bajari
Work done while at Amazon, US
G
Guido Imbens
Graduate School of Business and Department of Economics, Stanford University, US