Admissibility Breakdown in High-Dimensional Sparse Regression with L1 Regularization

2026-01-15

AI Summary

In high-dimensional sparse linear regression, the choice of the regularization parameter critically affects the mean squared prediction error performance of the Lasso. This work, within a non-asymptotic framework, is the first to explicitly characterize the tuning threshold beyond which the Lasso estimator becomes inadmissible, revealing the pivotal roles of the design matrix and noise structure in determining this inadmissibility. To address this limitation, the authors propose a Lasso–Ridge hybrid method whose regularization path strictly dominates several classical Lasso tuning strategies. The proposed approach achieves substantially improved prediction accuracy, offering a theoretically grounded and practically superior criterion for regularization selection in high-dimensional regression settings.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

The choice of the tuning parameter in the Lasso is central to its statistical performance in high-dimensional linear regression. Classical consistency theory identifies the rate of the Lasso tuning parameter, and numerous studies have established non-asymptotic guarantees. Nevertheless, the question of optimal tuning within a non-asymptotic framework has not yet been fully resolved. We establish tuning criteria above which the Lasso becomes inadmissible under mean squared prediction error. More specifically, we establish thresholds showing that certain classical tuning choices yield Lasso estimators strictly dominated by a simple Lasso-Ridge refinement. We also address how the structure of the design matrix and the noise vector influences the inadmissibility phenomenon.

Problem

• Lasso
• tuning parameter
• inadmissibility
• high-dimensional regression
• sparse regression

Innovation

• Lasso
• inadmissibility
• tuning parameter
• high-dimensional regression
• Lasso-Ridge refinement