Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration

2024-12-20

AI Summary

This work addresses core challenges in human-AI collaboration—preference understanding, domain-specific adaptation, and dynamic control allocation—by proposing Co-Gym, a collaborative reinforcement learning framework. Co-Gym enables asynchronous, three-way interaction among agent, human, and environment across three representative tasks: travel planning, tabular data analysis, and literature review. Its key innovations include a ternary collaboration modeling mechanism and a dual-dimension evaluation system assessing both process fidelity and outcome quality, quantified via multi-granular metrics including user win rate, interaction efficiency, and intent alignment. Employing a hybrid simulation–real-user evaluation paradigm, the study identifies critical boundaries where collaboration outperforms fully autonomous systems and pinpoints three fundamental bottlenecks: communication capability, contextual awareness, and control trade-off. Real-user experiments demonstrate that the collaborative agent achieves win rates of 86%, 74%, and 66% across the three tasks—significantly surpassing fully autonomous baselines.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Multiagent Systems: Coordination and Collaboration
• Game Theory and Economic Paradigms: Coordination and Collaboration

Application Category

• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Recent advancements in language models (LMs) have sparked growing interest in developing LM agents. While fully autonomous agents could excel in many scenarios, numerous use cases inherently require them to collaborate with humans due to humans' latent preferences, domain expertise, or need for control. To facilitate the study of human-agent collaboration, we present Collaborative Gym (Co-Gym), a general framework enabling asynchronous, tripartite interaction among agents, humans, and task environments. We instantiate Co-Gym with three representative tasks in both simulated and real-world conditions, and propose an evaluation framework that assesses both the collaboration outcomes and processes. Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance within those delivered cases, achieving win rates of 86% in Travel Planning, 74% in Tabular Analysis, and 66% in Related Work when evaluated by real users. However, our study also highlights significant challenges in developing collaborative agents, requiring advancements in core aspects of intelligence -- communication capabilities, situational awareness, and balancing autonomy and human control.

Problem

• Human-Robot Collaboration
• Preference Understanding
• Decision Control

Innovation

• Co-Gym
• Collaborative AI
• Human-AI Interaction