Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD

2026-01-15

AI Summary

Under the worst-case adversarial model, DP-SGD struggles to simultaneously achieve strong privacy guarantees and high model utility. This work establishes, for the first time within the f-differential privacy framework, an explicit lower bound on the privacy–utility trade-off for single-pass shuffled DP-SGD, revealing a fundamental limitation that prevents their simultaneous optimization. By integrating hypothesis testing curves, Poisson subsampling, and Gaussian noise mechanism analysis, we prove that if the noise multiplier σ is less than 1/√(2ln M), then the privacy leakage κ is at least 1/√8. Empirical results further demonstrate that enforcing this privacy constraint necessitates a noise level that significantly degrades model accuracy.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Differentially Private Stochastic Gradient Descent (DP-SGD) is the dominant paradigm for private training, but its fundamental limitations under worst-case adversarial privacy definitions remain poorly understood. We analyze DP-SGD in the $f$-differential privacy framework, which characterizes privacy via hypothesis-testing trade-off curves, and study shuffled sampling over a single epoch with $M$ gradient updates. We derive an explicit suboptimal upper bound on the achievable trade-off curve. This result induces a geometric lower bound on the separation $\kappa$ which is the maximum distance between the mechanism's trade-off curve and the ideal random-guessing line. Because a large separation implies significant adversarial advantage, meaningful privacy requires small $\kappa$. However, we prove that enforcing a small separation imposes a strict lower bound on the Gaussian noise multiplier $\sigma$, which directly limits the achievable utility. In particular, under the standard worst-case adversarial model, shuffled DP-SGD must satisfy $\sigma \ge \frac{1}{\sqrt{2\ln M}}$ $\quad\text{or}\quad$ $\kappa \ge\ \frac{1}{\sqrt{8}}\!\left(1-\frac{1}{\sqrt{4\pi\ln M}}\right)$, and thus cannot simultaneously achieve strong privacy and high utility. Although this bound vanishes asymptotically as $M \to \infty$, the convergence is extremely slow: even for practically relevant numbers of updates the required noise magnitude remains substantial. We further show that the same limitation extends to Poisson subsampling up to constant factors. Our experiments confirm that the noise levels implied by this bound leads to significant accuracy degradation at realistic training settings, thus showing a critical bottleneck in DP-SGD under standard worst-case adversarial assumptions.

Problem

• DP-SGD
• privacy-utility trade-off
• differential privacy
• adversarial privacy
• Gaussian noise

Innovation

• f-differential privacy
• DP-SGD
• privacy-utility trade-off
• shuffled sampling
• fundamental limits