Fundamental Limit of Discrete Distribution Estimation under Utility-Optimized Local Differential Privacy

2025-09-28

AI Summary

This work investigates the fundamental limits of utility-optimized local differential privacy (ULDP) for discrete distribution estimation—specifically, how to maximize inference accuracy for non-sensitive attributes while guaranteeing strict privacy for sensitive ones. We fully characterize the privacy–utility trade-off under ULDP by introducing an extremal ULDP mechanism family and a novel distribution decomposition technique, which reveals the structural properties of optimal mechanisms. Leveraging a generalized asymptotic Cramér–Rao lower bound, we derive a tight estimation error bound and propose the utility-optimized block design (uBD) mechanism coupled with a score-based linear estimator that achieves this bound. Theoretically and empirically, uBD significantly improves estimation accuracy for non-sensitive parameters. Our framework is the first to provide both tightness and achievability for ULDP, establishing an optimal, implementable solution for utility-aware private distribution estimation.

Technology Category

• Machine Learning: Privacy
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We study the problem of discrete distribution estimation under utility-optimized local differential privacy (ULDP), which enforces local differential privacy (LDP) on sensitive data while allowing more accurate inference on non-sensitive data. In this setting, we completely characterize the fundamental privacy-utility trade-off. The converse proof builds on several key ideas, including a generalized uniform asymptotic Cramér-Rao lower bound, a reduction showing that it suffices to consider a newly defined class of extremal ULDP mechanisms, and a novel distribution decomposition technique tailored to ULDP constraints. For the achievability, we propose a class of utility-optimized block design (uBD) schemes, obtained as nontrivial modifications of the block design mechanism known to be optimal under standard LDP constraints, while incorporating the distribution decomposition idea used in the converse proof and a score-based linear estimator. These results provide a tight characterization of the estimation accuracy achievable under ULDP and reveal new insights into the structure of optimal mechanisms for privacy-preserving statistical inference.

Problem

• Characterizing fundamental privacy-utility trade-offs under ULDP
• Proposing utility-optimized block design schemes for estimation
• Establishing tight accuracy bounds for discrete distribution estimation

Innovation

• Generalized uniform asymptotic Cramér-Rao lower bound
• Utility-optimized block design schemes modification
• Novel distribution decomposition technique ULDP