Development of Mental Models in Human-AI Collaboration: A Conceptual Framework

2025-10-09

AI Summary

Prior research predominantly focuses on AI agent design and collaborative architectures, overlooking the dynamic evolution of decision-makers’ mental models during sustained human-AI interaction. Method: This paper proposes an integrative socio-technical framework that systematically analyzes how three mechanisms—data contextualization, reasoning transparency, and performance feedback—drive mental model evolution. It innovatively defines three interdependent, dynamically evolving dimensions of mental models: domain cognition, information processing, and complementarity awareness. Drawing on conceptual modeling and interdisciplinary theoretical integration—specifically human factors engineering and AI-augmented decision-making theory—the framework formalizes a systematic design approach for human-AI co-evolution. Contribution/Results: The work advances the theoretical foundation of cognitive adaptation in human-AI collaboration and delivers actionable, mechanism-level design guidelines for optimizing socio-technical human-AI systems.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Multiagent Systems: Mechanism Design

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Artificial intelligence has become integral to organizational decision-making and while research has explored many facets of this human-AI collaboration, the focus has mainly been on designing the AI agent(s) and the way the collaboration is set up - generally assuming a human decision-maker to be "fixed". However, it has largely been neglected that decision-makers' mental models evolve through their continuous interaction with AI systems. This paper addresses this gap by conceptualizing how the design of human-AI collaboration influences the development of three complementary and interdependent mental models necessary for this collaboration. We develop an integrated socio-technical framework that identifies the mechanisms driving the mental model evolution: data contextualization, reasoning transparency, and performance feedback. Our work advances human-AI collaboration literature through three key contributions: introducing three distinct mental models (domain, information processing, complementarity-awareness); recognizing the dynamic nature of mental models; and establishing mechanisms that guide the purposeful design of effective human-AI collaboration.

Problem

• Addressing the evolution of decision-makers' mental models in human-AI collaboration
• Conceptualizing how collaboration design influences three interdependent mental models
• Establishing mechanisms to guide purposeful design of effective human-AI collaboration

Innovation

• Framework conceptualizes mental model evolution in collaboration
• Identifies data contextualization and reasoning transparency mechanisms
• Establishes performance feedback for purposeful AI collaboration design