Measuring Successful Cooperation in Human-AI Teamwork: Development and Validation of the Perceived Cooperativity and Teaming Perception Scales

2026-04-27

AI Summary

This study addresses the lack of reliable instruments for assessing subjective collaboration quality in human–AI teamwork. Drawing on joint activity theory and evolutionary cooperation theory, it develops and validates two novel, cross-agent and cross-context subjective scales: the Perceived Cooperativeness Scale (PCS) and the Teamness Perception Scale (TPS), which respectively measure perceived cooperativeness and sense of teaming. Through rigorous psychometric methods, the factorial structure, reliability, and construct validity of both scales were established across three studies (N = 409) and diverse experimental scenarios—including card games, interactions with large language models, and decision support systems. Findings demonstrate that the scales effectively differentiate partners exhibiting varying levels of collaborative quality, offering a unified and comparable tool for quantifying both human–AI and human–human collaboration.

Technology Category

• Humans and AI: Teamwork, Team formation
• Multiagent Systems: Teamwork
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

As human-AI cooperation becomes increasingly prevalent, reliable instruments for assessing the subjective quality of cooperative human-AI interaction are needed. We introduce two theoretically grounded scales: the Perceived Cooperativity Scale (PCS), grounded in joint activity theory, and the Teaming Perception Scale (TPS), grounded in evolutionary cooperation theory. The PCS captures an agent's perceived cooperative capability and practice within a single interaction sequence; the TPS captures the emergent sense of teaming arising from mutual contribution and support. Both scales were adapted for human-human cooperation to enable cross-agent comparisons. Across three studies (N = 409) encompassing a cooperative card game, LLM interaction, and a decision-support system, analyses of dimensionality, reliability, and validity indicated that both scales successfully differentiated between cooperation partners of varying cooperative quality and showed construct validity in line with expectations. The scales provide a basis for empirical investigation and system evaluation across a wide range of human-AI cooperation contexts.

Problem

• human-AI cooperation
• subjective quality
• cooperative interaction
• measurement scales
• team perception

Innovation

• Perceived Cooperativity Scale
• Teaming Perception Scale
• human-AI cooperation
• scale validation
• joint activity theory