Clustering and Median Aggregation Improve Differentially Private Inference

2025-06-05

AI Summary

In differentially private (DP) language model inference, uniform random sampling yields semantically inconsistent batches for heterogeneous-topic inputs, severely degrading private text quality. To address this, we propose a clustering-enhanced private inference framework: first, sensitive inputs are grouped via semantic k-means clustering to improve intra-batch topical coherence; second, leveraging data-dependent local sensitivity analysis, we replace conventional mean aggregation with private median aggregation to reduce noise overhead. This work is the first to jointly design input clustering and the private median mechanism, enabling post-hoc verifiable data-dependent DP guarantees. Experiments demonstrate significant improvements over state-of-the-art methods across MAUVE and downstream task performance, generating higher-quality synthetic text under lower privacy budgets (ε).

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Search and Optimization: Distributed Search

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Differentially private (DP) language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large language model (LLM) to produce a similar example. Multiple examples can be aggregated together to formally satisfy the DP guarantee. Prior work creates inference batches by sampling sensitive inputs uniformly at random. We show that uniform sampling degrades the quality of privately generated text, especially when the sensitive examples concern heterogeneous topics. We remedy this problem by clustering the input data before selecting inference batches. Next, we observe that clustering also leads to more similar next-token predictions across inferences. We use this insight to introduce a new algorithm that aggregates next token statistics by privately computing medians instead of averages. This approach leverages the fact that the median has decreased local sensitivity when next token predictions are similar, allowing us to state a data-dependent and ex-post DP guarantee about the privacy properties of this algorithm. Finally, we demonstrate improvements in terms of representativeness metrics (e.g., MAUVE) as well as downstream task performance. We show that our method produces high-quality synthetic data at significantly lower privacy cost than a previous state-of-the-art method.

Problem

• Improving DP text quality via clustering input data
• Enhancing privacy with median-based token aggregation
• Reducing privacy cost while maintaining synthetic data quality

Innovation

• Clustering input data for better batch selection
• Median aggregation for next token statistics
• Data-dependent DP guarantee with lower privacy cost