Gaze-informed Signatures of Trust and Collaboration in Human-Autonomy Teams

2024-09-27

arXiv.org

AI Summary

This study addresses trust formation and cognitive load mitigation in human–agent dynamic collaboration. We establish a multi-difficulty team task environment based on Overcooked AI and integrate eye-tracking metrics—including gaze allocation, revisit count, and scanpath dynamics—with hierarchical reinforcement learning (HRL) to systematically investigate how autonomous agent behavior influences human trust and coordination. We first identify a positive correlation between reduced gaze allocation and higher trust levels; propose “gaze revisit count” as a novel oculomotor indicator of agent predictability and environmental adaptability; and demonstrate that real-time eye-tracking signals can accurately characterize human contribution and enable agent self-adaptation. Experiments show that our adaptive agent significantly improves task balance, reduces collision rates, enhances coordination, and achieves higher subjective trust scores. Crucially, key oculomotor features robustly predict both trust level and human engagement.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Teamwork
• Computer Vision: Motion & Tracking

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In the evolving landscape of human-autonomy teaming (HAT), fostering effective collaboration and trust between human and autonomous agents is increasingly important. To explore this, we used the game Overcooked AI to create dynamic teaming scenarios featuring varying agent behaviors (clumsy, rigid, adaptive) and environmental complexities (low, medium, high). Our objectives were to assess the performance of adaptive AI agents designed with hierarchical reinforcement learning for better teamwork and measure eye tracking signals related to changes in trust and collaboration. The results indicate that the adaptive agent was more effective in managing teaming and creating an equitable task distribution across environments compared to the other agents. Working with the adaptive agent resulted in better coordination, reduced collisions, more balanced task contributions, and higher trust ratings. Reduced gaze allocation, across all agents, was associated with higher trust levels, while blink count, scan path length, agent revisits and trust were predictive of the humans contribution to the team. Notably, fixation revisits on the agent increased with environmental complexity and decreased with agent versatility, offering a unique metric for measuring teammate performance monitoring. These findings underscore the importance of designing autonomous teammates that not only excel in task performance but also enhance teamwork by being more predictable and reducing the cognitive load on human team members. Additionally, this study highlights the potential of eye-tracking as an unobtrusive measure for evaluating and improving human-autonomy teams, suggesting eye gaze could be used by agents to dynamically adapt their behaviors.

Problem

• Assess adaptive AI agents' teamwork performance in dynamic environments
• Measure eye-tracking signals linked to trust and collaboration changes
• Explore gaze patterns as metrics for human-autonomy team evaluation

Innovation

• Hierarchical reinforcement learning for adaptive agents
• Eye tracking to measure trust and collaboration
• Dynamic behavior adaptation based on gaze