On the Vulnerability of Text Sanitization

2024-10-22

arXiv.org

AI Summary

Existing privacy evaluations of text sanitization methods suffer from bias, primarily because empirical reconstruction attacks fail to characterize real-world risks. This work introduces the first theoretically optimal text reconstruction attack under differential privacy, rigorously deriving tight lower and upper bounds on its attack success rate (ASR). Leveraging information-theoretic principles and an optimal inference framework, we propose a practical attack combining token-level probabilistic modeling with fine-tuned language models. Experiments on SST-2 with ε = 4.0 show a 46.4% ASR improvement over state-of-the-art methods, exposing severe underestimation of privacy leakage in mainstream anonymization schemes. Our core contribution is establishing the first theory-driven text reconstruction attack paradigm, providing a verifiable, principled benchmark for evaluating differential privacy guarantees in textual data.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Text sanitization, which employs differential privacy to replace sensitive tokens with new ones, represents a significant technique for privacy protection. Typically, its performance in preserving privacy is evaluated by measuring the attack success rate (ASR) of reconstruction attacks, where attackers attempt to recover the original tokens from the sanitized ones. However, current reconstruction attacks on text sanitization are developed empirically, making it challenging to accurately assess the effectiveness of sanitization. In this paper, we aim to provide a more accurate evaluation of sanitization effectiveness. Inspired by the works of Palamidessi et al., we implement theoretically optimal reconstruction attacks targeting text sanitization. We derive their bounds on ASR as benchmarks for evaluating sanitization performance. For real-world applications, we propose two practical reconstruction attacks based on these theoretical findings. Our experimental results underscore the necessity of reassessing these overlooked risks. Notably, one of our attacks achieves a 46.4% improvement in ASR over the state-of-the-art baseline, with a privacy budget of epsilon=4.0 on the SST-2 dataset. Our code is available at: https://github.com/mengtong0110/On-the-Vulnerability-of-Text-Sanitization.

Problem

• Text Sanitization
• Privacy Protection
• Evaluation Accuracy

Innovation

• Privacy-preserving text cleaning evaluation
• Optimal attack strategy
• Real-world attack methods