Attune: A Self-Annotation Tool for Understanding Robot Operator Attention Profiles

2026-08-12

AI Summary

This work addresses the lack of empirical support in existing approaches for designing robot behaviors guided by operator attention in multi-robot supervision scenarios. It introduces Attune, a novel framework that leverages operators’ eye-tracking data as behavioral design cues for robots. By integrating eye-tracking, AI-assisted self-labeling, and visual analytics, Attune automatically identifies and abstracts key factors triggering attention shifts, producing interpretable summaries of attention patterns. The tool enables pre-deployment calibration of robot behaviors to align with individual operators’ attention profiles. User studies demonstrate that Attune effectively uncovers inter-individual differences in attention allocation and accurately characterizes each operator’s distinctive attentional traits.

Abstract

Deploying robot fleets in complex, real-world environments requires human operators to supervise multiple robots simultaneously. Managing operator attention is a fundamental challenge of designing multi-robot supervision interfaces, encompassing both feed layout and feed content (i.e., robot behavior design). Thus far, designers lack empirical guidance on the latter-how to change a robot's behavior to capture, sustain, or relinquish operator attention during multi-robot supervision. In our vision of the future, designers should be able to use this guidance to calibrate robot behavior to different operator attention profiles. Treating operator eye gaze as a robot behavior design clue, we created a pre-deployment elicitation tool called Attune. Attune automatically identifies when meaningful gaze shifts occur, provides AI assistance for annotating why shifts occurred, and outputs a summary of operator gaze patterns for operator review. We evaluated Attune through a user study in which participants annotated the visual triggers that drew their attention. Our findings unveil variation in observed gaze patterns and reveal how Attune helps characterize operator attention.

Problem

• multi-robot supervision
• operator attention
• robot behavior design
• attention profiles
• human-robot interaction

Innovation

• self-annotation
• operator attention
• eye gaze
• multi-robot supervision
• AI-assisted annotation