Indicating Robot Vision Capabilities with Augmented Reality

2025-11-05

AI Summary

Humans frequently misjudge robots’ field of view (FoV), forming inaccurate mental models that misalign with actual visual capabilities—leading to infeasible commands (e.g., requesting identification of out-of-view objects), especially when robots focus narrowly on tasks without active scene scanning. To address this, we propose four augmented reality (AR)-based FoV visualization methods, jointly representing FoV from both egocentric and task-space perspectives, and integrating physical affordances (e.g., deep-set eye sockets) with AR overlays. In a user study with 41 participants, we employed eye-tracking, cognitive load assessment, and task accuracy metrics. Results show the task-space cubic indicator achieves highest accuracy, while deep-set eye sockets significantly improve perceptual accuracy. All designs sustain high user confidence and low cognitive load. From these findings, we distill six evidence-based design principles for calibrating users’ mental models—establishing a novel paradigm for explainable human–robot collaboration.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Research indicates that humans can mistakenly assume that robots and humans have the same field of view (FoV), possessing an inaccurate mental model of robots. This misperception may lead to failures during human-robot collaboration tasks where robots might be asked to complete impossible tasks about out-of-view objects. The issue is more severe when robots do not have a chance to scan the scene to update their world model while focusing on assigned tasks. To help align humans'mental models of robots'vision capabilities, we propose four FoV indicators in augmented reality (AR) and conducted a user human-subjects experiment (N=41) to evaluate them in terms of accuracy, confidence, task efficiency, and workload. These indicators span a spectrum from egocentric (robot's eye and head space) to allocentric (task space). Results showed that the allocentric blocks at the task space had the highest accuracy with a delay in interpreting the robot's FoV. The egocentric indicator of deeper eye sockets, possible for physical alteration, also increased accuracy. In all indicators, participants'confidence was high while cognitive load remained low. Finally, we contribute six guidelines for practitioners to apply our AR indicators or physical alterations to align humans'mental models with robots'vision capabilities.

Problem

• Humans mistakenly assume robots share human field of view capabilities
• Inaccurate mental models lead to impossible task requests during collaboration
• Robots may fail when unable to scan scenes while performing tasks

Innovation

• AR indicators show robot field of view
• Allocentric blocks improve accuracy with delay
• Egocentric eye sockets increase accuracy physically