Not All Learnable Distribution Classes are Privately Learnable

2024-02-01

International Conference on Algorithmic Learning Theory

AI Summary

This paper addresses Ashtiani’s conjecture that learnability under total variation (TV) distance implies differentially private learnability. Method: We construct an explicit family of distributions that is learnable to constant TV error with finite samples in the non-private setting, yet provably unlearnable to the same accuracy under $(varepsilon,delta)$-differential privacy. Our analysis combines information-theoretic lower bounds, formal modeling within the private learning framework, and precise characterization of the sample complexity of TV-distance estimation under privacy constraints. Contribution/Results: We provide the first rigorous proof of a fundamental separation between classical and differentially private learnability. The result refutes Ashtiani’s conjecture and establishes that differential privacy imposes intrinsic limitations on statistical learning capacity—beyond mere sample-size overhead. This yields a critical theoretical criterion for the feasibility boundary of private learning and clarifies the inherent trade-off between privacy guarantees and statistical utility in distribution learning.

Technology Category

• Machine Learning: Privacy
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We give an example of a class of distributions that is learnable in total variation distance with a finite number of samples, but not learnable under $(varepsilon, delta)$-differential privacy. This refutes a conjecture of Ashtiani.

Problem

• Identifies a non-private learnable distribution class
• Shows it is not learnable under differential privacy
• Weakly refutes Ashtiani's conjecture

Innovation

• Example of non-private learnable distribution class
• Refutes Ashtiani's conjecture on privacy
• Compares learnable and private learnable distributions