Continual Release Moment Estimation with Differential Privacy

2025-02-10

AI Summary

In streaming data settings, existing differentially private (DP) moment estimation methods suffer from low accuracy and high noise. To address this, we propose a novel DP moment estimation framework that jointly leverages sensitivity analysis and the matrix mechanism. Our approach is the first to enable simultaneous private estimation of first-order (mean) and second-order (covariance) moments, with the covariance estimate incurring no additional privacy cost beyond that required for the mean. By optimizing sensitivity bounds and tailoring the matrix mechanism design, we significantly suppress estimation noise and improve statistical utility. Empirical evaluation on Gaussian density estimation and DP-Adam training of CIFAR-10 demonstrates that our method reduces mean and covariance estimation errors by 27%–41% over baseline approaches and accelerates model convergence by 1.8×, confirming its superior trade-off between privacy guarantees and statistical accuracy.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Stochastic Optimization
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We propose Joint Moment Estimation (JME), a method for continually and privately estimating both the first and second moments of data with reduced noise compared to naive approaches. JME uses the matrix mechanism and a joint sensitivity analysis to allow the second moment estimation with no additional privacy cost, thereby improving accuracy while maintaining privacy. We demonstrate JME's effectiveness in two applications: estimating the running mean and covariance matrix for Gaussian density estimation, and model training with DP-Adam on CIFAR-10.

Problem

• Estimates first and second moments privately
• Reduces noise in continual data estimation
• Applies to Gaussian density and model training

Innovation

• Joint Moment Estimation technique
• Matrix mechanism integration
• Zero additional privacy cost