🤖 AI Summary
Addressing the fundamental trade-off between robustness and accuracy in high-dimensional linear regression, this paper proposes an automated radius selection method grounded in Wasserstein distributionally robust optimization. For the first time under the high-dimensional asymptotic regime, we derive a convex–concave error characterization formula involving only four scalar variables, precisely capturing how estimation error varies with the robustness radius. This analytical expression eliminates the need for computationally expensive cross-validation, enabling theory-driven hyperparameter tuning. The theoretically predicted error closely matches empirical results, and the optimal radius selected by our method aligns with cross-validation outcomes—while reducing computational cost by over two orders of magnitude. Our core contribution is the establishment of an analytically tractable and numerically computable explicit model for robust estimation error, yielding the first automatic tuning framework for high-dimensional robust regression that simultaneously satisfies theoretical rigor and engineering practicality.
📝 Abstract
Distributionally robust optimization (DRO) has become a powerful framework for estimation under uncertainty, offering strong out-of-sample performance and principled regularization. In this paper, we propose a DRO-based method for linear regression and address a central question: how to optimally choose the robustness radius, which controls the trade-off between robustness and accuracy. Focusing on high-dimensional settings where the dimension and the number of samples are both large and comparable in size, we employ tools from high-dimensional asymptotic statistics to precisely characterize the estimation error of the resulting estimator. Remarkably, this error can be recovered by solving a simple convex-concave optimization problem involving only four scalar variables. This characterization enables efficient selection of the radius that minimizes the estimation error. In doing so, it achieves the same effect as cross-validation, but at a fraction of the computational cost. Numerical experiments confirm that our theoretical predictions closely match empirical performance and that the optimal radius selected through our method aligns with that chosen by cross-validation, highlighting both the accuracy and the practical benefits of our approach.
