From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Tasks

2025-04-03

AI Summary

This study addresses the risk of human cognitive passivity induced by generative AI in open-ended knowledge tasks. We propose a novel two-dimensional human-AI interaction evaluation framework—spanning exploration/exploitation and construction/damage—establishing, for the first time, a cognitively grounded, activity- and participation-oriented evaluation paradigm that overcomes the longstanding assessment challenges in open tasks: absence of canonical answers and difficulty in quantification. Leveraging interaction log modeling, cognitive behavior coding, pattern-based clustering, and mixed qualitative-quantitative analysis, we render collaborative cognition measurable, diagnosable, and optimizable—specifically quantifying “cognitive health.” Our findings rigorously delineate the boundary between *instrumental augmentation* (AI as a reasoning scaffold) and *substitutive erosion* (AI displacing core cognitive processes), thereby providing both theoretical foundations and actionable design principles for AI systems that actively preserve and enhance human reasoning capacity.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Humans and AI: Interaction Techniques and Devices
• Philosophy and Ethics of AI: Artificial General Intelligence

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The rise of Generative AI, and Large Language Models (LLMs) in particular, is fundamentally changing cognitive processes in knowledge work, raising critical questions about their impact on human reasoning and problem-solving capabilities. As these AI systems become increasingly integrated into workflows, they offer unprecedented opportunities for augmenting human thinking while simultaneously risking cognitive erosion through passive consumption of generated answers. This tension is particularly pronounced in open-ended tasks, where effective solutions require deep contextualization and integration of domain knowledge. Unlike structured tasks with established metrics, measuring the quality of human-LLM interaction in such open-ended tasks poses significant challenges due to the absence of ground truth and the iterative nature of solution development. To address this, we present a framework that analyzes interaction patterns along two dimensions: cognitive activity mode (exploration vs. exploitation) and cognitive engagement mode (constructive vs. detrimental). This framework provides systematic measurements to evaluate when LLMs are effective tools for thought rather than substitutes for human cognition, advancing theoretical understanding and practical guidance for developing AI systems that protect and augment human cognitive capabilities.

Problem

• Impact of LLMs on human reasoning in open-ended tasks
• Measuring human-LLM interaction quality without ground truth
• Balancing AI augmentation and cognitive erosion risks

Innovation

• Framework analyzes cognitive activity and engagement modes
• Measures human-LLM interaction in open-ended tasks
• Evaluates LLMs as tools for thought augmentation