Efficient Proposal-Test-Release for Minimax Optimal Estimation

2026-05-04

AI Summary

Differential privacy in statistical estimation often suffers from degraded accuracy due to the requirement of providing uniform guarantees across all datasets, including those containing outliers, which complicates sensitivity analysis. To address this challenge, this work proposes an efficient Proposal-Test-Release (ePTR) framework that replaces the intractable exact computations in classical PTR with a computationally efficient approximation of a non-sensitive subset and a lower bound on the Hellinger distance derived from Lipschitz continuity. This approach achieves minimax optimal estimation rates while strictly adhering to differential privacy. Empirical evaluations demonstrate that ePTR significantly outperforms existing differentially private baselines across Bayesian classification, linear regression, and nonparametric regression tasks, striking a superior balance between privacy preservation and estimation accuracy.

Abstract

Differential privacy (DP) is a rigorous framework that protects the participation of individuals in a dataset by limiting information leakage from released estimators. This creates a challenging setting for statisticians: DP must hold uniformly over all possible datasets, whereas statistical practice often downweights atypical or rare outcomes. The conceptual challenge is especially pronounced in sensitivity analysis, the key quantity governing the magnitude of DP noise and, consequently, estimator accuracy, because many estimators, including ordinary least squares for linear regression, exhibit markedly higher sensitivity on atypical datasets. Propose-Test-Release (PTR) is designed to address such cases, but its classical implementation requires computing the exact insensitive set and the dataset's Hellinger distance to that set, both of which are typically intractable. We introduce efficient PTR (ePTR), which replaces the exact insensitive set with a simpler subset and the exact Hellinger distance with a Lipschitz-based lower bound. This flexibility enables substantially simpler DP mechanisms that achieve rate-optimal accuracy in many settings. To illustrate, we study basic estimators for Bayes classification, linear regression, and nonparametric regression. We show that each can have high sensitivity on atypical datasets, yet admits intuitive ePTR-based designs. Empirically and theoretically, we compare ePTR against popular DP baselines and demonstrate improved accuracy while maintaining privacy guarantees.

Problem

• differential privacy
• sensitivity analysis
• Propose-Test-Release
• minimax optimal estimation
• statistical estimation

Innovation

• efficient PTR
• differential privacy
• minimax optimal estimation
• sensitivity analysis
• Lipschitz bound