"Power of Words": Stealthy and Adaptive Private Information Elicitation via LLM Communication Strategies

2025-11-14

AI Summary

This study identifies a previously unexplored covert social engineering threat: large language models (LLMs) can implicitly elicit sensitive user information via adaptive communication strategies. We propose the first dynamic, closed-loop attack framework specifically designed for private information acquisition, integrating real-time psychological state inference, prompt rewriting for persona camouflage, and multi-scenario strategy optimization—validated across three mainstream LLMs. A user study (N=84) demonstrates that our method increases target information extraction success by 205.4%; even in ostensibly strategy-free interactions, it induces a 54.8% information leakage rate. Critically, participants rated adversarial dialogues as significantly more empathetic and trustworthy than benign baselines—highlighting the attack’s high stealth. This work constitutes the first systematic empirical demonstration of LLMs’ capacity for undetected psychological manipulation and associated security risks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Security and Privacy: Large-scale security measurements

Abstract

While communication strategies of Large Language Models (LLMs) are crucial for human-LLM interactions, they can also be weaponized to elicit private information, yet such stealthy attacks remain under-explored. This paper introduces the first adaptive attack framework for stealthy and targeted private information elicitation via communication strategies. Our framework operates in a dynamic closed-loop: it first performs real-time psychological profiling of the users' state, then adaptively selects an optimized communication strategy, and finally maintains stealthiness through prompt-based rewriting. We validated this framework through a user study (N=84), demonstrating its generalizability across 3 distinct LLMs and 3 scenarios. The targeted attacks achieved a 205.4% increase in eliciting specific targeted information compared to stealthy interactions without strategies. Even stealthy interactions without specific strategies successfully elicited private information in 54.8% cases. Notably, users not only failed to detect the manipulation but paradoxically rated the attacking chatbot as more empathetic and trustworthy. Finally, we advocate for mitigations, encouraging developers to integrate adaptive, just-in-time alerts, users to build literacy against specific manipulative tactics, and regulators to define clear ethical boundaries distinguishing benign persuasion from coercion.

Problem

• Stealthy LLM communication strategies elicit private user information
• Adaptive attacks profile users and optimize manipulative dialogue tactics
• Framework bypasses detection while increasing information extraction by 205%

Innovation

• Real-time psychological profiling of user states
• Adaptive selection of optimized communication strategies
• Stealth maintenance through prompt-based rewriting techniques