An information theorist's tour of differential privacy

2025-10-11

AI Summary

This work establishes a systematic theoretical connection between differential privacy (DP) and information theory to quantitatively characterize the intrinsic information leakage of privacy-preserving mechanisms. Method: DP mechanisms are modeled as stochastic channels from input data to output analysis; information-theoretic measures—including mutual information and min-entropy leakage—are introduced and endowed with rigorous privacy semantics. By integrating probabilistic distribution perturbation analysis with channel capacity characterization, the paper derives quantitative relationships between the privacy budget ε and information leakage. Contribution/Results: It provides the first operational information-theoretic interpretation of DP from a channel coding perspective, revealing that ε-DP inherently enforces a worst-case information constraint. Moreover, it establishes computable information-theoretic bounds for the privacy–utility trade-off, thereby significantly strengthening the theoretical foundation and interpretability of DP.

Technology Category

• Machine Learning: Privacy
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Since being proposed in 2006, differential privacy has become a standard method for quantifying certain risks in publishing or sharing analyses of sensitive data. At its heart, differential privacy measures risk in terms of the differences between probability distributions, which is a central topic in information theory. A differentially private algorithm is a channel between the underlying data and the output of the analysis. Seen in this way, the guarantees made by differential privacy can be understood in terms of properties of this channel. In this article we examine a few of the key connections between information theory and the formulation/application of differential privacy, giving an ``operational significance'' for relevant information measures.

Problem

• Quantifying privacy risks in sensitive data analysis
• Modeling differential privacy as information theory channels
• Establishing operational significance for information measures

Innovation

• Differential privacy measures risk via probability distributions
• Private algorithms act as channels between data and output
• Information theory provides operational significance for privacy guarantees