CAPE: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

2025-05-09

AI Summary

To address the risk of sensitive user data leakage in large language model (LLM) inference services, existing differential privacy (DP) approaches struggle to simultaneously ensure strong privacy guarantees, low inference latency, and high generation utility. This paper proposes a context-aware DP prompt perturbation mechanism: first, it introduces a novel token similarity metric grounded in a hybrid utility function, enabling context-sensitive semantic preservation; second, it designs a bucketed random sampling strategy to mitigate long-tail distribution issues inherent in large perturbation spaces. Extensive experiments across multiple datasets demonstrate that, under identical privacy budgets (ε), our method improves BLEU and ROUGE scores by 12.3% over state-of-the-art baselines, while increasing inference latency by less than 8%. The approach thus achieves a significantly improved three-way trade-off among privacy, utility, and efficiency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen. Existing solutions primarily rely on privacy-enhancing technologies to mitigate such risks, facing the trade-off among efficiency, privacy, and utility. To narrow this gap, we propose Cape, a context-aware prompt perturbation mechanism based on differential privacy, to enable efficient inference with an improved privacy-utility trade-off. Concretely, we introduce a hybrid utility function that better captures the token similarity. Additionally, we propose a bucketized sampling mechanism to handle large sampling space, which might lead to long-tail phenomenons. Extensive experiments across multiple datasets, along with ablation studies, demonstrate that Cape achieves a better privacy-utility trade-off compared to prior state-of-the-art works.

Problem

• Prevent sensitive data leakage in LLM inference services
• Improve privacy-utility trade-off in prompt perturbation
• Handle large sampling space and token similarity efficiently

Innovation

• Context-aware prompt perturbation with differential privacy
• Hybrid utility function for better token similarity
• Bucketized sampling to handle large sampling space