"When to Hand Off, When to Work Together": Expanding Human-Agent Co-Creative Collaboration through Concurrent Interaction

2026-03-02

AI Summary

This work addresses the challenge that current AI agents struggle to interpret users’ concurrent interaction intents on shared artifacts, thereby limiting dynamic co-creation. To overcome this, we propose CLEO—a collaborative intelligent agent grounded in mixed-initiative interaction principles—that dynamically switches among delegation, guidance, and collaboration modes by recognizing user concurrent behaviors in real time. We introduce the first collaborative model capable of real-time intent interpretation, identifying five behavioral patterns, six triggering mechanisms, and four enabling factors, and implement a decision framework comprising six interactive loops. Based on 214 rounds of interactions with professional designers, we quantitatively analyze mode usage (70.1% delegation, 28.5% guidance, 31.8% collaboration) and release design guidelines alongside a labeled dataset to support future research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Intelligent User Interfaces
• Multiagent Systems: Modeling other Agents

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Human collaborators coordinate dynamically through process visibility and workspace awareness, yet AI agents typically either provide only final outputs or expose read-only execution processes (e.g., planning, reasoning) without interpreting concurrent user actions on shared artifacts. Building on mixed-initiative interaction principles, we explore whether agents can achieve collaborative context awareness -- interpreting concurrent user actions on shared artifacts and adapting in real-time. Study 1 (N=10 professional designers) revealed that process visibility enabled reasoning about agent actions but exposed conflicts when agents could not distinguish feedback from independent work. We developed CLEO, which interprets collaborative intent and adapts in real-time. Study 2 (N=10, two-day with stimulated recall interviews) analyzed 214 turns, identifying five action patterns, six triggers, and four enabling factors explaining when designers choose delegation (70.1%), direction (28.5%), or concurrent work (31.8%). We present a decision model with six interaction loops, design implications, and an annotated dataset.

Problem

• human-agent collaboration
• co-creative interaction
• concurrent interaction
• collaborative context awareness
• shared artifacts

Innovation

• collaborative context awareness
• concurrent interaction
• mixed-initiative interaction
• intent interpretation
• human-agent co-creation