Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond

2026-06-16

AI Summary

This study addresses the gap in existing research, which predominantly relies on expert-crafted hypothetical questions, by offering the first systematic analysis of real user inquiries in the domain of digital security and privacy (S&P). Leveraging 14,727 authentic user dialogues from the WildChat dataset, the authors employ topic modeling, multi-turn repeated prompting, and human evaluation to characterize users’ actual S&P concerns and rigorously assess the quality and consistency of responses from leading large language models. Findings reveal that commercial models—such as GPT-5.5—deliver “sufficiently good” answers on 98% of advisory prompts, substantially outperforming open-source counterparts like Llama-4 (47%), yet exhibit notable inconsistencies across conversational turns. This work provides an empirical foundation for understanding genuine user needs and evaluating model reliability in real-world S&P contexts.

Abstract

Large language models (LLMs) are widely used to fulfill users' information needs; users ask LLMs about the weather, pose educational questions, and consult them for legal assistance. One particularly understudied area is digital security and privacy (S&P), where users may seek LLMs' help on how to secure their online accounts or protect their computers from cyber attacks. To the best of our knowledge, no prior study has collected or analyzed the S&P questions users ask LLMs; prior research on LLM response quality relied on expert-authored S&P misconceptions or FAQs rather than user queries. Drawing from WildChat, a dataset of 3.2M user-LLM conversations collected in the wild, our study identifies 14,727 S&P prompts and categorizes them into nine categories covering a wide range of S&P topics. From the S&P prompts, we sampled 450 and performed a thematic analysis to characterize the S&P questions users ask LLMs. Separate from the thematic analysis, we curated 270 advice-seeking S&P prompts, where users ask for recommendations, guidance, or specific S&P information. We measured LLM response quality and consistency when posing the prompt to LLMs 10 times. We found that commercial LLMs outperform open-weight models (GPT 5.5 provided "good enough" responses on 98% of prompts; Llama 4 on 47%). However, among prompts that received high-quality responses on average, commercial models sometimes produce contradictory responses across runs, risking confusing or misleading users.

Problem

• Security and Privacy
• Large Language Models
• User Queries
• Response Quality
• LLM Safety

Innovation

• security and privacy
• large language models
• user queries
• response consistency
• empirical analysis