Quadratic Objective Perturbation: Curvature-Based Differential Privacy

2026-05-07

AI Summary

This work addresses the limitations of traditional linear objective perturbation, which relies on gradient boundedness assumptions that are often violated in modern machine learning models. The authors propose Quadratic Objective Perturbation (QOP), a novel mechanism that leverages the curvature of the loss function as the cornerstone of privacy guarantees. By introducing randomized quadratic perturbations, QOP induces strong convexity and algorithmic stability under interpolation conditions, thereby eliminating the need for gradient boundedness. Privacy sensitivity is controlled through the spectral properties of the perturbation matrix, enabling differential privacy even with approximate solutions and achieving $(\varepsilon, \delta)$-differential privacy under weaker assumptions. Both theoretical analysis and empirical evaluations demonstrate that QOP yields a superior privacy-utility trade-off compared to existing linear objective perturbation methods and provides guaranteed bounds on empirical excess risk.

Abstract

Objective perturbation is a standard mechanism in differentially private empirical risk minimization. In particular, Linear Objective Perturbation (LOP) enforces privacy by adding a random linear term, while strong convexity and stability are ensured by an additional deterministic quadratic term. However, this approach requires the strong assumption of bounded gradients of the loss function, which excludes many modern machine learning models. In this work, we introduce Quadratic Objective Perturbation (QOP), which perturbs the objective with a random quadratic form. This perturbation induces strong convexity and enforces stability of the problem through curvature, thereby enabling privacy and allowing sensitivity to be controlled through spectral properties of the perturbation rather than assumptions on the gradients. As a result, we obtain $(\varepsilon, δ)$-differential privacy under weaker assumptions, in the interpolation regime. Furthermore, we extend the analysis to account for approximate solutions, showing that privacy guarantees are preserved under inexact solves. Additionally, we derive utility guarantees in terms of empirical excess risk, and provide a theoretical and numerical comparison to LOP, highlighting the advantages of curvature-based perturbations. Finally, we discuss algorithmic aspects and show that the resulting problems can be solved efficiently using modern splitting schemes.

Problem

• differential privacy
• empirical risk minimization
• objective perturbation
• gradient boundedness
• curvature-based perturbation

Innovation

• Quadratic Objective Perturbation
• Curvature-Based Privacy
• Differential Privacy
• Empirical Risk Minimization
• Strong Convexity