Help or Hindrance: Understanding the Impact of Robot Communication in Action Teams

2025-06-10

AI Summary

This study investigates the collaborative efficacy of multimodal human–computer communication (speech + vision) in time-critical operational teams, such as medical emergency training. We developed a Remote-Controlled Cart (RCC) prototype integrating speech-based task prompts, LED status indicators, and behavior-triggered logic to support task reminders and object search guidance in a laboratory-simulated emergency scenario. Evaluation employed a dual-dimensional approach: subjective workload (NASA-TLX) and usability (UMUX-Lite), complemented by objective behavioral performance metrics. Results demonstrate, for the first time, a synergistic mechanism wherein speech modality improves search efficiency while visual feedback enhances task memory retention. Relative to the no-feedback baseline, the multimodal configuration significantly reduced team cognitive load (p < 0.01), increased perceived usability (+32%) and usefulness (+28%). These findings empirically validate the practical value and distinctive contribution of multimodal feedback in high-tempo, mission-critical emergency response contexts.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The human-robot interaction (HRI) field has recognized the importance of enabling robots to interact with teams. Human teams rely on effective communication for successful collaboration in time-sensitive environments. Robots can play a role in enhancing team coordination through real-time assistance. Despite significant progress in human-robot teaming research, there remains an essential gap in how robots can effectively communicate with action teams using multimodal interaction cues in time-sensitive environments. This study addresses this knowledge gap in an experimental in-lab study to investigate how multimodal robot communication in action teams affects workload and human perception of robots. We explore team collaboration in a medical training scenario where a robotic crash cart (RCC) provides verbal and non-verbal cues to help users remember to perform iterative tasks and search for supplies. Our findings show that verbal cues for object search tasks and visual cues for task reminders reduce team workload and increase perceived ease of use and perceived usefulness more effectively than a robot with no feedback. Our work contributes to multimodal interaction research in the HRI field, highlighting the need for more human-robot teaming research to understand best practices for integrating collaborative robots in time-sensitive environments such as in hospitals, search and rescue, and manufacturing applications.

Problem

• How robots communicate effectively in time-sensitive team environments
• Impact of multimodal robot cues on team workload and perception
• Optimizing robot feedback for task coordination in medical scenarios

Innovation

• Multimodal robot communication in action teams
• Verbal and non-verbal cues for task assistance
• Reducing workload with effective robot feedback