Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inference

2025-09-15

AI Summary

This study uncovers a novel inference-based privacy risk: large language models (LLMs) can implicitly infer users’ sensitive attributes from ordinary text. To investigate user perception biases and mitigation efficacy, we conducted a mixed-methods survey (N=217) and controlled comparative experiments evaluating textual rewriting strategies. Results reveal that users consistently underestimate LLMs’ inference capabilities; semantic interventions—such as abstraction and ambiguity introduction—significantly outperform conventional paraphrasing. Human-authored de-identification succeeded in only 28% of cases—marginally better than Rescriber but markedly inferior to ChatGPT-generated anonymized text. Our core contribution is the first systematic characterization of privacy-preserving behavioral patterns “in-the-loop” under inference threats, establishing a risk-aware, interactive privacy-enhancement design paradigm for human–LLM collaboration.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Humans and AI: AI for Accessibility

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) such as ChatGPT can infer personal attributes from seemingly innocuous text, raising privacy risks beyond memorized data leakage. While prior work has demonstrated these risks, little is known about how users estimate and respond. We conducted a survey with 240 U.S. participants who judged text snippets for inference risks, reported concern levels, and attempted rewrites to block inference. We compared their rewrites with those generated by ChatGPT and Rescriber, a state-of-the-art sanitization tool. Results show that participants struggled to anticipate inference, performing a little better than chance. User rewrites were effective in just 28% of cases - better than Rescriber but worse than ChatGPT. We examined our participants' rewriting strategies, and observed that while paraphrasing was the most common strategy it is also the least effective; instead abstraction and adding ambiguity were more successful. Our work highlights the importance of inference-aware design in LLM interactions.

Problem

• Users struggle to anticipate implicit LLM inference risks
• User rewrites are ineffective at blocking attribute inference
• Current sanitization tools underperform compared to LLM-generated text

Innovation

• Surveying user strategies for LLM inference mitigation
• Comparing human rewrites with AI sanitization tools
• Recommending abstraction and ambiguity for privacy protection