When Are LLM Inferences Acceptable? User Reactions and Control Preferences for Inferred Personal Information

2026-05-11

AI Summary

This study addresses growing privacy concerns around large language models (LLMs) inferring sensitive user attributes from conversations without explicit disclosure. To investigate users’ actual perceptions and control preferences regarding such inferences, the authors designed and deployed Reflective Layer, a visualization tool that revealed 215 model-generated inferences to 18 ChatGPT users. Combining log analysis, LLM-based inference extraction, interactive visualizations, and mixed-methods interviews and surveys, the research empirically demonstrates that users do not uniformly perceive LLM inferences as risky; instead, their acceptance is context-dependent, shaped by inference accuracy, usage scenario, and data destination. Discomfort primarily arises from inaccurate or misaligned uses, and participants strongly reject the use of such inferences by advertisers or third parties—challenging the prevailing assumption that inference inherently constitutes a privacy risk.

Abstract

Ask ChatGPT about vacation planning, and it may infer your income. Ask it about medication, and it may infer your medical history. Because such inferences can expose more information than users intend to reveal, prior work argues that they are a defining privacy risk of LLM-based systems. Yet prior work has mostly shown that LLMs can make potentially violating inferences, not how users experience those inferences nor what controls users may want governing their use. We built the Reflective Layer, a visualization tool that surfaces example unstated inferences from users' own ChatGPT histories, and used it in a mixed-methods study with 18 regular ChatGPT users evaluating 215 surfaced inferences from their own conversations. Counterintuitively, participants reacted more strongly with curiosity and interest rather than distress and concern. Discomfort arose mainly when inferences felt misrepresentative of the user or misaligned with expected use. Participants were also markedly less comfortable with advertisers and third-party applications using those inferences than with platform providers. These findings suggest that the acceptability of LLM inferences is governed not only by its content, but by context-sensitive norms around how they are generated, retained within the platform, and transmitted beyond it.

Problem

• LLM inferences
• privacy risk
• user reactions
• personal information
• control preferences

Innovation

• LLM inferences
• privacy perceptions
• user control
• Reflective Layer
• context-sensitive norms