User Prompting Strategies and ChatGPT Contextual Adaptation Shape Conversational Information-Seeking Experiences

2025-09-29

AI Summary

This study investigates how ordinary users employ prompting strategies in conversational information seeking (CIS) with ChatGPT and how AI responses adapt contextually—particularly across scientific, health, and policy topics differing in controversy level. Method: A nationally representative mixed-methods study integrates dialogue log analysis, computational text complexity metrics (e.g., lexical diversity, syntactic depth), and user attitude surveys. Contribution/Results: Only a small subset of highly educated, Democratic-leaning users deploy advanced prompting techniques—revealing a novel dimension of the digital divide. ChatGPT exhibits active contextual adaptation in controversial topics, significantly increasing response cognitive complexity and external citation density. Although such high-complexity responses reduce immediate perceived preference, they foster more positive topic attitudes. This is the first empirical demonstration of co-adaptation between user prompting behavior and AI response style, offering foundational insights for CIS interface design and AI literacy interventions.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Humans and AI: Intelligent User Interfaces
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Conversational AI, such as ChatGPT, is increasingly used for information seeking. However, little is known about how ordinary users actually prompt and how ChatGPT adapts its responses in real-world conversational information seeking (CIS). In this study, a nationally representative sample of 937 U.S. adults engaged in multi-turn CIS with ChatGPT on both controversial and non-controversial topics across science, health, and policy contexts. We analyzed both user prompting strategies and the communication styles of ChatGPT responses. The findings revealed behavioral signals of digital divide: only 19.1% of users employed prompting strategies, and these users were disproportionately more educated and Democrat-leaning. Further, ChatGPT demonstrated contextual adaptation: responses to controversial topics contain more cognitive complexity and more external references than to non-controversial topics. Notably, cognitively complex responses were perceived as less favorable but produced more positive issue-relevant attitudes. This study highlights disparities in user prompting behaviors and shows how user prompts and AI responses together shape information-seeking with conversational AI.

Problem

• Analyzes how users prompt ChatGPT and its contextual adaptation in information seeking
• Investigates digital divide in prompting strategies among different demographic groups
• Examines how ChatGPT responses shape user perceptions and attitudes on controversial topics

Innovation

• Analyzed user prompting strategies in ChatGPT interactions
• Measured ChatGPT cognitive complexity adaptation to topics
• Linked user demographics with prompting behavior disparities