Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation

2026-08-10

AI Summary

This work addresses a critical gap in existing privacy-preserving approaches, which predominantly focus on explicit identifiers while overlooking the re-identification risks posed by self-disclosed content lacking personally identifiable information (PII) but containing quasi-identifiers. To mitigate this, we propose the Probabilistic Privacy-Aware Delegation (PCD) framework, which uniquely incorporates k-anonymity as an auxiliary metric within PCD tasks. Leveraging large language models (LLMs), our method enables probabilistic k-anonymity estimation for more comprehensive privacy risk assessment and reduction. We introduce PUPA-SD, the first dataset capturing natural user self-disclosure behaviors, and implement a local LLM-mediated architecture that achieves optimal privacy–utility trade-offs on Llama-3.2-3B. Experiments demonstrate that our approach significantly enhances response quality in unseen dialogues, whereas smaller models struggle to balance privacy and utility effectively.

Abstract

Recent work on protecting privacy during user-LLM interactions often focuses on direct, explicit identifiers: the personally-identifiable information (PII) captured by standard detectors. One such approach is Privacy-Conscious Delegation (PCD), where a local LLM acts as an intermediary. However, privacy risk does not stem solely from explicit identifiers but also PII-free self-disclosures, leaving users identifiable through combinations of quasi-identifying traits. We investigate a probabilistic variant of PCD, where we augment its objectives with an LLM-driven probabilistic estimation of k-anonymity. To facilitate this, we first create the PUPA-SD dataset, which contains naturalistic user queries with self-disclosure. Our preliminary results indicate that optimizing PAPILLON on PUPA-SD improves quality on unseen conversations across a variety of local models and produces the best privacy-utility balance for Llama-3.2-3B, while smaller models struggle to jointly optimize quality and privacy. We propose k-anonymity as a useful auxiliary metric for tackling PCD.

Problem

• privacy risk
• self-disclosure
• quasi-identifiers
• k-anonymity
• LLM query delegation

Innovation

• probabilistic k-anonymity
• Privacy-Conscious Delegation
• quasi-identifiers
• self-disclosure
• LLM privacy risk