Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents

2025-02-22

AI Summary

This work addresses the problem of unintentional disclosure of sensitive information by users during interactions with large language models (LLMs), stemming from insufficient privacy awareness. We formally introduce the concept of *contextual privacy*—retaining only the minimal, task-relevant context necessary to fulfill the user’s intent. To this end, we propose a lightweight, locally deployable real-time prompt sanitization framework that operates as an intermediary layer prior to LLM input. It integrates user intent modeling, context boundary detection, relevance classification, and dynamic rewriting to identify and reconstruct privacy-sensitive content. Evaluated on the ShareGPT dataset, our approach reduces sensitive information exposure by 62% while preserving 98.3% task completion accuracy. Our key contributions include: (1) the first formal definition of contextual privacy; (2) a client-side, real-time sanitization architecture; and (3) empirical evidence revealing indirect information leakage—a previously underexplored privacy threat in LLM interactions.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Conversational agents are increasingly woven into individuals' personal lives, yet users often underestimate the privacy risks involved. The moment users share information with these agents (e.g., LLMs), their private information becomes vulnerable to exposure. In this paper, we characterize the notion of contextual privacy for user interactions with LLMs. It aims to minimize privacy risks by ensuring that users (sender) disclose only information that is both relevant and necessary for achieving their intended goals when interacting with LLMs (untrusted receivers). Through a formative design user study, we observe how even"privacy-conscious"users inadvertently reveal sensitive information through indirect disclosures. Based on insights from this study, we propose a locally-deployable framework that operates between users and LLMs, and identifies and reformulates out-of-context information in user prompts. Our evaluation using examples from ShareGPT shows that lightweight models can effectively implement this framework, achieving strong gains in contextual privacy while preserving the user's intended interaction goals through different approaches to classify information relevant to the intended goals.

Problem

• Addressing privacy risks in LLM interactions
• Minimizing sensitive information disclosure
• Developing a framework for contextual privacy

Innovation

• Contextual privacy framework
• Locally-deployable interface
• Information relevance classification