Optimal Variance and Covariance Estimation under Differential Privacy in the Add-Remove Model and Beyond

2025-09-05

AI Summary

Under the add-remove differential privacy model—where dataset size is private—the estimation of variance and covariance is more challenging and less studied than under the swap model. This paper proposes the first minimax-optimal variance/covariance estimation method for this setting. Our approach introduces a novel moment-release framework based on the Bessel mechanism and, for the first time, employs Bernstein basis functions to characterize the sensitivity of statistical moments, thereby substantially improving utility. We theoretically establish that our estimator achieves the minimax lower bound in the high-privacy regime. Empirical evaluations demonstrate that it consistently outperforms existing mechanisms in variance and covariance estimation, as well as in downstream statistical tasks—delivering both strong privacy guarantees and high estimation accuracy. Moreover, the framework supports scalable, multi-task deployment.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

In this paper, we study the problem of estimating the variance and covariance of datasets under differential privacy in the add-remove model. While estimation in the swap model has been extensively studied in the literature, the add-remove model remains less explored and more challenging, as the dataset size must also be kept private. To address this issue, we develop efficient mechanisms for variance and covariance estimation based on the emph{Bézier mechanism}, a novel moment-release framework that leverages Bernstein bases. We prove that our proposed mechanisms are minimax optimal in the high-privacy regime by establishing new minimax lower bounds. Moreover, beyond worst-case scenarios, we analyze instance-wise utility and show that the Bézier-based estimator consistently achieves better utility compared to alternative mechanisms. Finally, we demonstrate the effectiveness of the Bézier mechanism beyond variance and covariance estimation, showcasing its applicability to other statistical tasks.

Problem

• Estimating variance and covariance under differential privacy
• Addressing challenges in the add-remove model with private dataset size
• Developing optimal mechanisms using Bézier framework for statistical estimation

Innovation

• Bezier mechanism for variance estimation
• Minimax optimal in high-privacy regime
• Instance-wise utility improvement over alternatives