On the optimality of antithetic randomization for cross-validation

2026-08-08

AI Summary

This study addresses the randomized construction of training–test folds in cross-validation, aiming to simultaneously control estimation bias and reducible variance. The authors propose an optimal design based on antithetic randomization and elucidate how bias and variance depend on the marginal and joint distributions of data partitions. Theoretically, they establish that for smooth estimators, antithetic randomization with a specific negative correlation structure is necessary and sufficient for bounded variance; for nonsmooth estimators, this approach substantially improves the variance convergence rate. By integrating joint Gaussian constructions, control variates, and asymptotic variance analysis, they develop a minimax-optimal randomization scheme that ensures variance remains bounded even as bias vanishes asymptotically.

Abstract

In the classical normal means problem, independent train--test folds can be constructed by perturbing the data with normal randomization. Averaging over $K$ such folds yields a cross-validation estimator whose bias depends on the marginal distribution of the randomization variables, while its variance depends on their joint distribution. This raises the questions: which joint law is optimal, and how to construct the corresponding randomization scheme? We show that: (i) for smooth estimators, antithetic randomization with pairwise correlation $ρ=-1/(K-1)$ is necessary and sufficient for the reducible variance due to randomization to remain bounded as the bias vanishes; (ii) a general construction yields a class of antithetic schemes, within which the jointly normal scheme is minimax optimal; and (iii) for non-smooth estimators with finitely many jump discontinuities, antithetic randomization improves the asymptotic rate of the reducible variance, while a simple control variate restores bounded variance when the discontinuities are known.

Problem

• cross-validation
• randomization
• antithetic sampling
• variance reduction
• normal means problem

Innovation

• antithetic randomization
• cross-validation
• variance reduction
• minimax optimality
• normal means problem