Mind the Gap: Mixtures of Gaussians in Approximate Differential Privacy

2026-05-27

AI Summary

In the moderate-to-low privacy regime (i.e., small $(\varepsilon, \delta)$), existing Gaussian mechanisms are significantly suboptimal due to excessive noise injection. This work proposes a hybrid Gaussian noise mechanism that constructs a convex combination of multiple Gaussian distributions with identical variances but distinct means, adaptively tuning both the means and mixing weights using sensitivity information. It presents the first systematic construction and analysis of a Gaussian mixture-based perturbation scheme satisfying $(\varepsilon, \delta)$-differential privacy. The authors derive tight variance conditions and an efficient algorithm that substantially reduce both L1 and L2 utility loss in the low-privacy regime, markedly narrowing the performance gap with the theoretically optimal mechanism and achieving near-optimal accuracy.

Abstract

We design a class of additive noise mechanisms that satisfy \((\varepsilon, δ)\)-differential privacy (DP) for scalar, real-valued query functions with known sensitivities, with a particular focus on moderate and low-privacy regimes. These mechanisms, which we call \textit{mixture mechanisms}, are constructed by mixing multiple Gaussian distributions that share the same variance but differ in their means and mixture weights. The resulting distributions can be interpreted as convex combinations of a zero-mean Gaussian (as used in the analytic Gaussian mechanism) and additional Gaussians whose means depend on the sensitivity of the query function. We derive tight conditions on the variances required for \((\varepsilon, δ)\)-DP and provide efficient algorithms to compute them. Compared to the analytic Gaussian mechanism, our mechanisms yield substantially lower expected noise amplitudes (\(l_1\)-loss) and variances (\(l_2\)-loss for zero-mean distributions). In the low-privacy regime that motivates our design, our mechanisms approach optimality, mitigating nearly all of the optimality gap of the analytic Gaussian mechanism.

Problem

• differential privacy
• Gaussian mechanism
• additive noise
• privacy-utility tradeoff
• optimality gap

Innovation

• mixture mechanisms
• approximate differential privacy
• Gaussian mixtures
• optimality gap
• noise minimization