Enhancing Differentially Private Mechanisms via Empirical Bayes

2026-06-19

AI Summary

This work addresses a central challenge in differential privacy: enhancing algorithmic utility without compromising privacy guarantees or incurring excessive computational complexity. The authors propose a post-processing denoising method grounded in empirical Bayes estimation, which effectively reduces mean squared error using only the outputs of Gaussian differential privacy mechanisms. To the best of our knowledge, this is the first systematic application of the empirical Bayes framework to differential privacy post-processing. The approach significantly improves utility without altering the underlying privacy mechanism, offering both simplicity and broad applicability. Empirical evaluations demonstrate consistent performance gains over existing differentially private algorithms across diverse tasks, including histogram release, principal component analysis, and linear regression.

Abstract

Differential privacy (DP) has become the gold standard for ensuring the privacy protection of machine learning and statistical algorithms in recent decades. A plethora of algorithms and methods have been developed to enhance the utility of DP algorithms while maintaining the same level of DP. However, these are often overly complex or computationally ineffective. We propose a novel approach focusing on denoising the output of the simple additive Gaussian mechanism by adopting the idea of \textit{empirical Bayes estimation}. We highlight that the empirical Bayes approach can reduce the mean-squared error solely by taking the output of the Gaussian mechanism as input. Our numerical studies show that this simple yet powerful approach can be applied to improve upon various statistical problems, including histogram release, principal component analysis, and linear regression, often outperforming existing private algorithms.

Problem

• differential privacy
• utility enhancement
• mean-squared error
• Gaussian mechanism
• empirical Bayes

Innovation

• empirical Bayes
• differential privacy
• Gaussian mechanism
• denoising
• mean-squared error