Robust and Well-conditioned Sparse Estimation for High-dimensional Covariance Matrices

2025-12-29

AI Summary

Robust sparse estimation of high-dimensional covariance matrices faces three interrelated challenges: difficulty in guaranteeing positive definiteness, sparsity degradation due to post-hoc corrections, and uncontrolled condition numbers. Method: We propose the first method that explicitly incorporates a condition-number constraint into a robust adaptive thresholding framework. Using convex optimization and a provably convergent alternating direction algorithm, our approach jointly ensures positive definiteness, sparsity, and numerical stability. Contribution/Results: We establish theoretical minimax optimal convergence rate under the Frobenius norm. Experiments on both synthetic and real-world datasets demonstrate that our estimator consistently yields positive definite, sparse, and well-conditioned (low condition number) covariance matrices. Its numerical stability matches or surpasses that of eigenvalue truncation, while requiring fewer hyperparameters and offering greater practical utility.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Estimating covariance matrices with high-dimensional complex data presents significant challenges, particularly concerning positive definiteness, sparsity, and numerical stability. Existing robust sparse estimators often fail to guarantee positive definiteness in finite samples, while subsequent positive-definite correction can degrade sparsity and lack explicit control over the condition number. To address these limitations, we propose a novel robust and well-conditioned sparse covariance matrix estimator. Our key innovation is the direct incorporation of a condition number constraint within a robust adaptive thresholding framework. This constraint simultaneously ensures positive definiteness, enforces a controllable level of numerical stability, and preserves the desired sparse structure without resorting to post-hoc modifications that compromise sparsity. We formulate the estimation as a convex optimization problem and develop an efficient alternating direction algorithm with guaranteed convergence. Theoretically, we establish that the proposed estimator achieves the minimax optimal convergence rate under the Frobenius norm. Comprehensive simulations and real-data applications demonstrate that our method consistently produces positive definite, well-conditioned, and sparse estimates, and achieves comparable or superior numerical stability to eigenvalue-bound methods while requiring less tuning parameters.

Problem

• Ensures positive definiteness in high-dimensional covariance estimation
• Maintains sparsity while controlling condition number directly
• Achieves robust numerical stability without post-hoc corrections

Innovation

• Direct condition number constraint in robust thresholding
• Convex optimization with efficient alternating direction algorithm
• Achieves minimax optimal convergence rate theoretically