Block Designs that Provide Optimal Power in the Cochran-Mantel-Haenszel Test

2025-07-10

AI Summary

This paper investigates asymptotic statistical power optimization of the Cochran–Mantel–Haenszel (CMH) test under local alternatives in Fisher’s randomized block design. Leveraging asymptotic statistical theory and randomization inference, we derive and characterize a “balance condition” that ensures asymptotic optimality of the CMH test: it holds automatically when potential outcomes are orderable or under Greevy et al. (2004) pair-matching designs (under mild regularity conditions); moreover, any orderable potential outcomes converge to balance as the number of blocks diverges. We prove that designs satisfying this condition achieve asymptotically optimal power. Small-sample simulations reveal second-order effects favoring fewer blocks, while in multi-covariate settings, pair-matching effectively approximates the balance condition, substantially improving finite-sample efficiency. The core contribution is the first formal, interpretable balance criterion linking CMH test power to block structure, accompanied by actionable design guidelines for practitioners.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Causal Learning

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We consider the asymptotic power performance under local alternatives of the Cochran-Mantel-Haenszel test. Our setting is non-traditional: we investigate randomized experiments that assign subjects via Fisher's blocking design. We show that blocking designs that satisfy a certain balance condition are asymptotically optimal. When the potential outcomes can be ordered, the balance condition is met for all blocking designs with number of blocks going to infinity. More generally, we prove that the pairwise matching design of Greevy et al. (2004) satisfies the balance condition under mild assumptions. In smaller sample sizes, we show a second order effect becomes operational thereby making blocking designs with a smaller number optimal. In practical settings with many covariates, we recommend pairwise matching for its ability to approximate the balance condition.

Problem

• Optimizing power in Cochran-Mantel-Haenszel test
• Investigating blocking designs for randomized experiments
• Determining optimal balance conditions for blocking

Innovation

• Fisher's blocking design for randomized experiments
• Pairwise matching satisfies balance condition
• Optimal power in Cochran-Mantel-Haenszel test