Admissibility of Completely Randomized Trials: A Large-Deviation Approach

2025-06-05

AI Summary

This paper investigates best-arm identification in multi-armed trials, focusing on whether fully randomized designs are justifiably admissible. Methodologically, it employs large-deviation theory to define a novel “efficiency index” that rigorously quantifies statistical efficiency. Theoretically, it proves that for three or more arms, certain batch-elimination adaptive designs strictly dominate non-adaptive full randomization—thereby refuting the admissibility of full randomization in multi-armed settings. This resolves the second open problem posed by Qin (2022). Furthermore, the paper derives a simple, verifiable sufficient condition under which batch-elimination designs strictly dominate full randomization across all parameter configurations, achieving exponential improvement in sample complexity. The results establish fundamental limits on the statistical efficiency of non-adaptive designs and provide principled guidance for designing efficient sequential experiments in multi-armed decision-making.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Search and Optimization: Adversarial Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

When an experimenter has the option of running an adaptive trial, is it admissible to ignore this option and run a non-adaptive trial instead? We provide a negative answer to this question in the best-arm identification problem, where the experimenter aims to allocate measurement efforts judiciously to confidently deploy the most effective treatment arm. We find that, whenever there are at least three treatment arms, there exist simple adaptive designs that universally and strictly dominate non-adaptive completely randomized trials. This dominance is characterized by a notion called efficiency exponent, which quantifies a design's statistical efficiency when the experimental sample is large. Our analysis focuses on the class of batched arm elimination designs, which progressively eliminate underperforming arms at pre-specified batch intervals. We characterize simple sufficient conditions under which these designs universally and strictly dominate completely randomized trials. These results resolve the second open problem posed in Qin [2022].

Problem

• Is non-adaptive trial admissible versus adaptive trial
• Best-arm identification problem with multiple treatment arms
• Efficiency exponent quantifies statistical efficiency dominance

Innovation

• Adaptive designs dominate non-adaptive trials
• Batched arm elimination improves efficiency
• Efficiency exponent quantifies statistical performance