Convex Optimization with Local Label Differential Privacy: Tight Bounds in All Privacy Regimes

2026-05-11

AI Summary

This work addresses stochastic convex optimization (SCO) under local label differential privacy (L-LDP), where only labels are locally perturbed while features remain public. The paper proposes the first non-interactive algorithm that achieves excess risk upper bounds of \(O(\sqrt{K/(\varepsilon n)})\) under high privacy (\(\varepsilon \leq 1\)) and \(O(\sqrt{K/(e^\varepsilon n)})\) under moderate privacy (\(1 < \varepsilon \leq \ln K\)). Notably, it reduces the dependence on the label space size \(K\) from the previously known \(O(K)\) to the information-theoretically tight \(O(\sqrt{K})\). This improvement is attained through a novel integration of LDP mechanisms, convex optimization theory, and information-theoretic lower bound analysis, substantially enhancing the privacy-utility trade-off.

Abstract

We study the problem of Stochastic Convex Optimization (SCO) under the constraint of local Label Differential Privacy (L-LDP). In this setting, the features are considered public, but the corresponding labels are sensitive and must be randomized by each user locally before being sent to an untrusted analyzer. Prior work for SCO under L-LDP (Ghazi et al., 2021) established an excess population risk bound with a \emph{linear} dependence on the size of the label space, $K$: $O\left({\frac{K}{ε\sqrt{n}}}\right)$ in the high-privacy regime ($ε\leq 1$) and $O\left({\frac{K}{e^ε \sqrt{n}}}\right)$ in the medium-privacy regime ($1 \leq ε\leq \ln K$). This left open whether this linear cost is fundamental to the L-LDP model. In this note, we resolve this question. First, we present a novel and efficient non-interactive L-LDP algorithm that achieves an excess risk of $O\left({\sqrt{\frac{K}{εn}}}\right)$ in the high-privacy regime ($ε\leq 1$) and $O\left({\sqrt{\frac{K}{e^ε n}}}\right)$ in the medium-privacy regime ($1 \leq ε\leq \ln K$). This quadratically improves the dependency on the label space size from $O(K)$ to $O(\sqrt{K})$. Second, we prove a matching information-theoretic lower bound across all privacy regimes for any sufficiently large $n$.

Problem

• Stochastic Convex Optimization
• Local Label Differential Privacy
• Excess Risk
• Label Space
• Privacy Regimes

Innovation

• Local Label Differential Privacy
• Stochastic Convex Optimization
• Excess Risk Bound
• Non-interactive Algorithm
• Information-theoretic Lower Bound