Concept-Aware Privacy Mechanisms for Defending Embedding Inversion Attacks

2026-02-06

AI Summary

This work addresses the vulnerability of text embeddings to inversion attacks that compromise privacy, noting that existing differential privacy approaches often degrade utility by injecting excessive noise due to their assumption of uniform sensitivity across all embedding dimensions. To overcome this limitation, the authors propose SPARSE, a novel framework that enables fine-grained, user-defined concept-level privacy protection. SPARSE employs differentiable mask learning to identify sensitive dimensions and applies targeted perturbations via an ellipsoidal noise mechanism based on Mahalanobis distance. This approach breaks away from conventional uniform noise injection, significantly reducing privacy leakage across six datasets, three embedding models, and multiple attack scenarios, while consistently outperforming state-of-the-art differential privacy methods in downstream task performance.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Text embeddings enable numerous NLP applications but face severe privacy risks from embedding inversion attacks, which can expose sensitive attributes or reconstruct raw text. Existing differential privacy defenses assume uniform sensitivity across embedding dimensions, leading to excessive noise and degraded utility. We propose SPARSE, a user-centric framework for concept-specific privacy protection in text embeddings. SPARSE combines (1) differentiable mask learning to identify privacy-sensitive dimensions for user-defined concepts, and (2) the Mahalanobis mechanism that applies elliptical noise calibrated by dimension sensitivity. Unlike traditional spherical noise injection, SPARSE selectively perturbs privacy-sensitive dimensions while preserving non-sensitive semantics. Evaluated across six datasets with three embedding models and attack scenarios, SPARSE consistently reduces privacy leakage while achieving superior downstream performance compared to state-of-the-art DP methods.

Problem

• embedding inversion attacks
• privacy
• text embeddings
• differential privacy
• sensitive attributes

Innovation

• concept-aware privacy
• embedding inversion attacks
• differentiable mask learning
• Mahalanobis mechanism
• text embeddings