Lasso Universality Under Linearly Dependent Covariates in the Sparse Regime

2026-08-08

AI Summary

This work addresses the lack of theoretical guarantees for the universality of Lasso estimation in high-dimensional sparse regression when the design matrix exhibits complex, non-Gaussian linear dependencies across both rows and columns. We establish, for the first time, a Gaussian universality theorem for Lasso under a broad class of simultaneous row-and-column dependence structures that significantly relaxes the independence or specialized dependency assumptions prevalent in existing literature. By integrating tools from high-dimensional probability, random matrix theory, and universality arguments, we rigorously demonstrate that the behavior of Lasso under such intricate dependencies remains asymptotically equivalent to that under Gaussian designs. Extensive numerical experiments across diverse sparsity configurations corroborate the theoretical findings, confirming both the validity and robustness of our results.

Abstract

Throughout the last decade, Gaussian universality has been widely studied for high-dimensional estimation problems. Most of the literature focuses on i.i.d. sensing matrices or accounts for special forms of dependence, such as block dependence or other specific row/column dependencies. More general simultaneous row and column mixing has not yet been fully studied. In this paper, we focus on that setting. We prove a Gaussian universality theorem for the lasso in the sparse regime, where the non- Gaussian covariates have linearly dependent rows and columns. To the best of our knowledge, our setting permits a broader simultaneous row and column dependence structure than those treated in much of the prior universality literature. Numerical illustrations for various sparse profiles support the universality claims of this paper.

Problem

• Lasso
• Gaussian universality
• linearly dependent covariates
• sparse regime
• high-dimensional estimation

Innovation

• Gaussian universality
• Lasso
• linearly dependent covariates
• sparse regime
• row-column dependence