🤖 AI Summary
Cross-validation data splitting in high-dimensional covariance matrix estimation lacks rigorous finite-sample theoretical foundations, particularly under hold-out validation.
Method: We derive the first closed-form analytical expression for the estimation error under the white inverse Wishart population model, integrating high-dimensional statistics, random matrix theory, and asymptotic spectral analysis.
Contribution/Results: We establish that the optimal train-test ratio scales as $Theta(sqrt{p})$, where $p$ is the dimension; this scaling holds exactly for finite samples and unifies hold-out and $k$-fold cross-validation in the high-dimensional asymptotic limit—both converge to the optimal error bound of the nonlinear shrinkage estimator. Our work provides the first analytically tractable and empirically verifiable theoretical framework for cross-validation data partitioning in covariance estimation, resolving the longstanding gap in finite-sample performance characterization of cross-validation for this fundamental problem.
📝 Abstract
Cross-validation is a statistical tool that can be used to improve large covariance matrix estimation. Although its efficiency is observed in practical applications, the theoretical reasons behind it remain largely intuitive, with formal proofs currently lacking. To carry on analytical analysis, we focus on the holdout method, a single iteration of cross-validation, rather than the traditional $k$-fold approach. We derive a closed-form expression for the estimation error when the population matrix follows a white inverse Wishart distribution, and we observe the optimal train-test split scales as the square root of the matrix dimension. For general population matrices, we connected the error to the variance of eigenvalues distribution, but approximations are necessary. Interestingly, in the high-dimensional asymptotic regime, both the holdout and $k$-fold cross-validation methods converge to the optimal estimator when the train-test ratio scales with the square root of the matrix dimension.