Enforcing Demographic Coherence: A Harms Aware Framework for Reasoning about Private Data Release

2025-02-04

AI Summary

Differential privacy (DP) imposes overly stringent constraints that often degrade data utility, and its formal definitions exhibit a disconnect from practical privacy attack risks. Method: This paper proposes *demographic consistency* as a necessary condition for privacy protection—designed to mitigate individual re-identification risks arising from violations of population-level statistical regularities. We formulate attack-informed group consistency as an explicit privacy constraint, proving it is strictly weaker than DP, thereby permitting higher data utility while enabling privacy risk quantification without requiring explicit adversary specifications. We develop a general-purpose release framework based on confidence-score predictors and theoretically establish that all DP mechanisms satisfy demographic consistency; moreover, we construct practical algorithms satisfying demographic consistency but violating DP. Results: Empirical evaluation confirms the feasibility and utility advantages of the proposed approach over conventional DP mechanisms.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The technical literature about data privacy largely consists of two complementary approaches: formal definitions of conditions sufficient for privacy preservation and attacks that demonstrate privacy breaches. Differential privacy is an accepted standard in the former sphere. However, differential privacy's powerful adversarial model and worst-case guarantees may make it too stringent in some situations, especially when achieving it comes at a significant cost to data utility. Meanwhile, privacy attacks aim to expose real and worrying privacy risks associated with existing data release processes but often face criticism for being unrealistic. Moreover, the literature on attacks generally does not identify what properties are necessary to defend against them. We address the gap between these approaches by introducing demographic coherence, a condition inspired by privacy attacks that we argue is necessary for data privacy. This condition captures privacy violations arising from inferences about individuals that are incoherent with respect to the demographic patterns in the data. Our framework focuses on confidence rated predictors, which can in turn be distilled from almost any data-informed process. Thus, we capture privacy threats that exist even when no attack is explicitly being carried out. Our framework not only provides a condition with respect to which data release algorithms can be analysed but suggests natural experimental evaluation methodologies that could be used to build practical intuition and make tangible assessment of risks. Finally, we argue that demographic coherence is weaker than differential privacy: we prove that every differentially private data release is also demographically coherent, and that there are demographically coherent algorithms which are not differentially private.

Problem

• Addressing privacy-utility trade-offs
• Introducing demographic coherence condition
• Analyzing data release algorithms

Innovation

• Demographic coherence for privacy
• Confidence rated predictors framework
• Weaker than differential privacy