🤖 AI Summary
This paper investigates the performance of the difference-in-means estimator in two-arm randomized experiments under continuous, binary, proportion, and survival endpoints, considering both equal and unequal allocation within Neyman and superpopulation modeling frameworks. Methodologically, it integrates randomization inference, minimax analysis, asymptotic statistics, and NP-hardness arguments, complemented by Monte Carlo simulations. The study establishes, for the first time, that Fisher’s blocked design is asymptotically optimal under Kapelner’s tail criterion. It systematically characterizes the theoretical boundaries between complete randomization (Neyman model) and deterministic perfect balance (superpopulation model), identifying blocked design as the optimal compromise. Both theoretical analysis and simulation results consistently demonstrate that blocking substantially reduces estimation error and robustly outperforms complete randomization and pairwise matching across all endpoint types and allocation ratios.
📝 Abstract
We consider the performance of the difference-in-means estimator in a two-arm randomized experiment under common experimental endpoints such as continuous (regression), incidence, proportion and survival. We examine performance under both equal and unequal allocation to treatment groups and we consider both the Neyman randomization model and the population model. We show that in the Neyman model, where the only source of randomness is the treatment manipulation, there is no free lunch: complete randomization is minimax for the estimator's mean squared error. In the population model, where each subject experiences response noise with zero mean, the optimal design is the deterministic perfect-balance allocation. However, this allocation is generally NP-hard to compute and moreover, depends on unknown response parameters. When considering the tail criterion of Kapelner et al. (2021), we show the optimal design is less random than complete randomization and more random than the deterministic perfect-balance allocation. We prove that Fisher's blocking design provides the asymptotically optimal degree of experimental randomness. Theoretical results are supported by simulations in all considered experimental settings.
