Analysis of Mutual and Referential Human and Robot Gazes in a Collaborative Word Association Game

2026-07-13

AI Summary

This study investigates the mechanisms by which a robot’s gaze behavior influences human visual attention and confirmatory looking during cognitively demanding collaborative tasks. Participants engaged in a word-association game with an LLM-driven NAO robot under two conditions: mutual gaze and referential gaze. For the first time in a cognitively intensive setting, the work systematically examines how robotic gaze affects human confirmatory looking behavior. A fine-grained multimodal analysis integrating eye-tracking data, speech segments, and areas of interest reveals that while the robot’s gaze direction does not affect participants’ initial fixation latency on proposed words, utterances containing explicit confirmation requests significantly increase the frequency of human gaze returns to the robot. These findings suggest that linguistic cues may dominate over nonverbal gaze signals in shaping interactive attention dynamics.

Abstract

Robot gaze is a major component of human-robot dialogue coordination. Most studies of gaze in human-robot dialogue focus on face-to-face social conversations, but little is known about gaze in demanding task-focused interactions. In this paper, we investigate how the gaze of a robot game partner affects human visual attention and if humans tend to direct confirmation-seeking gazes towards the robot. In our study, we let participants play a collaborative word association game with a NAO robot acting as an embodied, LLM-driven conversational partner. Our experiments are conducted under two conditions, which implement mutual and referential gazes of the robot respectively. We record participants' gaze using eye tracking glasses and analyze the interactions using gaze coordinates, speech segments, key events and areas of interests. We find that robot gaze orientation does not affect the time to first fixation on words the robot proposed. We also find that participants gaze more often at the robot when their dialogue line contains confirmation requests, compared to when it does not. Our results indicate (likely also due to the cognitively demanding nature of the game) that the verbal aspect of this task overshadows the effects of referential robot gaze. These findings offer valuable insights for designing and validating robot gaze and turn-taking behavior in collaborative tasks which require coordination and efficient communication.

Problem

• human-robot gaze
• collaborative task
• visual attention
• confirmation-seeking
• word association game

Innovation

• robot gaze
• large language model (LLM)
• collaborative task
• eye tracking
• human-robot interaction