The Regularization Parameter: Sparse Precision Matrix Estimation

2026-07-07

AI Summary

This work addresses the challenge of selecting regularization parameters in sparse precision matrix estimation under high-dimensional, small-sample settings. The authors propose a closed-form, matrix-valued regularization parameter that eliminates the need for cross-validation. Derived from the first-order optimality conditions of the ℓ₁-regularized Gaussian maximum likelihood estimator, the method directly determines the parameter by modeling the probability that nonzero entries satisfy these conditions across resampled data. Theoretical analysis establishes its asymptotic and sparsistency properties. Empirical evaluations on both synthetic and real-world datasets—including gene expression microarrays and neuroimaging data—demonstrate that the proposed approach matches cross-validation in estimation accuracy, achieves superior support recovery, and reduces computational time by several orders of magnitude.

Abstract

Sparse precision matrix estimation provides an interpretable and computationally efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data. A recurring challenge is appropriately selecting the regularization parameter that controls estimator sparsity and strikes a balance between underfitting and overfitting. We propose a closed-form, matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions of the $\ell_1$-regularized Gaussian maximum-likelihood estimator. By prescribing the probability that each nonzero entry of the estimator satisfies its optimality condition under resampling, we eliminate the need for cross-validation. The resulting regularization parameter is shown to attain asymptotic scaling properties that, under standard conditions, provide consistency and sparsistency of the estimator. On synthetic Gaussian and non-Gaussian datasets, as well as real-world gene microarray and neuroimaging applications, the proposed approach achieves estimation accuracy comparable to cross-validation, delivers superior support recovery, and reduces runtime by several orders of magnitude.

Problem

• sparse precision matrix
• regularization parameter
• high-dimensional data
• sparsity
• model selection

Innovation

• sparse precision matrix
• regularization parameter
• closed-form solution
• sparsistency
• support recovery