Optimal Additive Noise Mechanisms for Differential Privacy

2025-04-20

AI Summary

This paper addresses the design of noise mechanisms for differential privacy, proposing the first unified framework that explicitly incorporates the Rényi parameter α into the optimization objective. It jointly optimizes continuous and discrete noise distributions to minimize Rényi differential privacy (RDP) under a given cost constraint, thereby strengthening (ε, δ)-differential privacy guarantees. The key contribution lies in reformulating the infinite-dimensional distributional optimization problem as a tractable finite-dimensional convex program and developing a preconditioned gradient descent algorithm for its solution. Compared to fixed-variance Gaussian and Laplace mechanisms, the proposed approach delivers significantly improved privacy guarantees in moderate-composition regimes. Numerical experiments consistently demonstrate superior performance over standard baselines across diverse settings.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We propose a unified optimization framework for designing continuous and discrete noise distributions that ensure differential privacy (DP) by minimizing R'enyi DP, a variant of DP, under a cost constraint. R'enyi DP has the advantage that by considering different values of the R'enyi parameter $alpha$, we can tailor our optimization for any number of compositions. To solve the optimization problem, we reduce it to a finite-dimensional convex formulation and perform preconditioned gradient descent. The resulting noise distributions are then compared to their Gaussian and Laplace counterparts. Numerical results demonstrate that our optimized distributions are consistently better, with significant improvements in $(varepsilon, delta)$-DP guarantees in the moderate composition regimes, compared to Gaussian and Laplace distributions with the same variance.

Problem

• Design optimal noise for differential privacy
• Minimize Rényi DP under cost constraints
• Compare performance with Gaussian and Laplace

Innovation

• Unified optimization framework for DP noise
• Minimizes Rényi DP under cost constraints
• Preconditioned gradient descent for convex formulation