Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration

2025-09-22

AI Summary

Traditional intelligent systems are predominantly designed as tools rather than collaborative partners, lacking essential capabilities to support human collaboration. Method: Addressing the applicability of foundational HCI/CSCW principles to human–LLM agent collaboration, we developed the first CSCW-theory-grounded, configurable human–agent collaboration research platform. Its modular architecture enables precise control of interaction variables, standardization of experimental protocols, and participatory cognitive walkthroughs; it reproduces the classic Shape Factory task and supports a 16-participant user study. Contribution/Results: Evaluated by five HCI researchers, the platform demonstrates strong usability and extensibility. This work advances human–AI collaboration from a “tool-use” paradigm toward a “joint cognition” paradigm, establishing both a methodological foundation and infrastructural support for systematic, empirical investigation of human–agent collaboration.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Intelligent Robots: Human-Robot Interaction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Intelligent systems have traditionally been designed as tools rather than collaborators, often lacking critical characteristics that collaboration partnerships require. Recent advances in large language model (LLM) agents open new opportunities for human-LLM-agent collaboration by enabling natural communication and various social and cognitive behaviors. Yet it remains unclear whether principles of computer-mediated collaboration established in HCI and CSCW persist, change, or fail when humans collaborate with LLM agents. To support systematic investigations of these questions, we introduce an open and configurable research platform for HCI researchers. The platform's modular design allows seamless adaptation of classic CSCW experiments and manipulation of theory-grounded interaction controls. We demonstrate the platform's effectiveness and usability through two case studies: (1) re-implementing the classic human-human-collaboration task Shape Factory as a between-subject human-agent-collaboration experiment with 16 participants, and (2) a participatory cognitive walkthrough with five HCI researchers to refine workflows and interfaces for experiment setup and analysis.

Problem

• Exploring whether human-computer collaboration principles apply to human-LLM-agent partnerships
• Developing a configurable platform to systematically study human-agent collaboration dynamics
• Adapting classic CSCW experiments to investigate human-LLM interaction patterns

Innovation

• Configurable platform for human-agent collaboration research
• Modular design enabling adaptation of CSCW experiments
• Validated through case studies with human participants