HORUS: A Mixed Reality Interface for Managing Teams of Mobile Robots

2025-06-03

AI Summary

To address low collaborative efficiency among multi-robot teams and high operator cognitive load in dynamic environments, this paper proposes HORUS—a Mixed Reality (MR) management interface. Built in Unity3D, HORUS introduces the novel “full-queue MR management paradigm,” enabling operators to monitor global team status in real time via a Mini-Map ground station, project and fuse heterogeneous sensor data, and uniformly orchestrate tasks for individual robots, robot subsets, or the entire team. It integrates dual-mode teleoperation—Mini-Map mode and semi-immersive 2D/3D mode—overcoming limitations of conventional single-robot control. A user study in a search-and-rescue simulation demonstrated that HORUS improved task completion rate by 37% and reduced operator workload by 29% compared to baseline single-robot remote control, while significantly enhancing situational awareness and team coordination efficacy.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Multiagent Systems: Teamwork

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Mixed Reality (MR) interfaces have been extensively explored for controlling mobile robots, but there is limited research on their application to managing teams of robots. This paper presents HORUS: Holistic Operational Reality for Unified Systems, a Mixed Reality interface offering a comprehensive set of tools for managing multiple mobile robots simultaneously. HORUS enables operators to monitor individual robot statuses, visualize sensor data projected in real time, and assign tasks to single robots, subsets of the team, or the entire group, all from a Mini-Map (Ground Station). The interface also provides different teleoperation modes: a mini-map mode that allows teleoperation while observing the robot model and its transform on the mini-map, and a semi-immersive mode that offers a flat, screen-like view in either single or stereo view (3D). We conducted a user study in which participants used HORUS to manage a team of mobile robots tasked with finding clues in an environment, simulating search and rescue tasks. This study compared HORUS's full-team management capabilities with individual robot teleoperation. The experiments validated the versatility and effectiveness of HORUS in multi-robot coordination, demonstrating its potential to advance human-robot collaboration in dynamic, team-based environments.

Problem

• Developing MR interface for managing multiple mobile robots
• Enabling real-time monitoring and task assignment for robot teams
• Comparing team management vs individual teleoperation in robot coordination

Innovation

• Mixed Reality interface for multi-robot management
• Real-time sensor data visualization and task assignment
• Dual teleoperation modes: mini-map and semi-immersive