Score Attack: A Lower Bound Technique for Optimal Differentially Private Learning

2023-03-13

arXiv.org

AI Summary

This work addresses the fundamental challenge of characterizing minimax optimal risk for parameter estimation under differential privacy. Conventional information-theoretic approaches fail in high-dimensional and nonparametric settings due to their inability to capture the interplay between privacy constraints and structural model assumptions. To overcome this limitation, we introduce the *score attack*—a novel framework that integrates score statistics with tracing attacks—yielding a general, computationally tractable lower bound for estimation under privacy constraints. Unlike prior methods, the score attack is dimension- and model-agnostic. We derive tight (up to logarithmic factors) minimax lower bounds for three canonical problems: high-dimensional sparse generalized linear models, the Bradley–Terry–Luce pairwise comparison model, and nonparametric regression over Sobolev spaces. These results establish both the broad applicability and statistical optimality of the proposed framework.

Abstract

Achieving optimal statistical performance while ensuring the privacy of personal data is a challenging yet crucial objective in modern data analysis. However, characterizing the optimality, particularly the minimax lower bound, under privacy constraints is technically difficult. To address this issue, we propose a novel approach called the score attack, which provides a lower bound on the differential-privacy-constrained minimax risk of parameter estimation. The score attack method is based on the tracing attack concept in differential privacy and can be applied to any statistical model with a well-defined score statistic. It can optimally lower bound the minimax risk of estimating unknown model parameters, up to a logarithmic factor, while ensuring differential privacy for a range of statistical problems. We demonstrate the effectiveness and optimality of this general method in various examples, such as the generalized linear model in both classical and high-dimensional sparse settings, the Bradley-Terry-Luce model for pairwise comparisons, and nonparametric regression over the Sobolev class.

Problem

• Lower bound for optimal differentially private learning
• Minimax risk estimation under privacy constraints
• Applicable to various statistical models

Innovation

• Score attack for privacy lower bounds
• Applies to models with score statistics
• Optimal minimax risk estimation guarantee