When AI Becomes Hard to Understand: Cognitive Demands in Real-World Human-AI Conversations

2026-09-15

AI Summary

研究分析了84,000多条ChatGPT和Gemini对话，探讨AI响应在实际对话中何时变得难以理解，并提出了一种对话复杂度预算的概念来优化用户、任务和交互的响应配置。

Abstract

Generative AI increasingly supports complex financial and health decisions, yet we know little about when its responses become difficult to process in real-world dialogue. We analyse more than 84,000 ChatGPT and Gemini conversations, using repeated prompting and clarification following misunderstanding as behavioural indicators of cognitive difficulty. We find that response characteristics such as length, readability and lexical diversity do not have fixed relationships with conversational difficulty; instead, their relationships depend on how they combine. Most notably, greater lexical diversity was associated with less repeated prompting in shorter responses, but this association weakened as response length increased, a pattern that replicated across financial and health conversations. We propose a conversational complexity budget to conceptualise these interdependencies: the demands associated with one response characteristic may depend on those accompanying it. The resulting design challenge is how to configure response complexity for the particular user, task and interaction.

Problem

• Cognitive Demands
• Human-AI Conversations
• Response Characteristics

Innovation

• conversational complexity budget
• lexical diversity
• response length