Minimax and Adaptive Covariance Matrix Estimation under Differential Privacy

2026-03-20

AI Summary

This study addresses the problem of optimal and adaptive estimation of high-dimensional banded covariance matrices under differential privacy constraints. The work proposes a novel block-wise tridiagonal differentially private covariance estimator that achieves minimax-optimal convergence rates in both operator and Frobenius norms, without requiring prior knowledge of the decay parameter. Key innovations include the first establishment of a van Trees inequality tailored to differentially private estimation, which reveals a polynomial dependence of privacy-induced error on dimensionality, and the development of a hierarchical tridiagonal adaptive estimation framework that integrates Bayesian priors with privacy-preserving mechanisms. Both theoretical analysis and empirical experiments confirm the optimality of the proposed method, quantify the accuracy loss due to privacy protection, and precisely characterize the fundamental trade-off between privacy and statistical accuracy.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Stochastic Optimization
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The covariance matrix plays a fundamental role in the analysis of high-dimensional data. This paper studies minimax and adaptive estimation of high-dimensional bandable covariance matrices under differential privacy constraints. We propose a novel differentially private blockwise tridiagonal estimator that achieves minimax-optimal convergence rates under both the operator norm and the Frobenius norm. In contrast to the non-private setting, the privacy-induced error exhibits a polynomial dependence on the ambient dimension, revealing a substantial additional cost of privacy. To establish optimality, we develop a new differentially private van Trees inequality and construct carefully designed prior distributions to obtain matching minimax lower bounds. The proposed private van Trees inequality applies more broadly to general private estimation problems and is of independent interest. We further introduce an adaptive estimator that attains the optimal rate up to a logarithmic factor without prior knowledge of the decay parameter, based on a novel hierarchical tridiagonal approach. Numerical experiments corroborate the theoretical results and illustrate the fundamental privacy-accuracy trade-off.

Problem

• covariance matrix
• differential privacy
• minimax estimation
• adaptive estimation
• high-dimensional statistics

Innovation

• differential privacy
• covariance matrix estimation
• minimax optimality
• adaptive estimation
• van Trees inequality