Validating Virtual Reality for Studying Multimodal Human-Robot Interaction in Socially Aware Robot Navigation

2026-07-10

AI Summary

This study systematically investigates whether virtual reality (VR) can effectively replicate the multimodal interaction dynamics between humans and robots during social navigation in the real world. By constructing an immersive VR environment and conducting controlled experiments alongside a physical motion-capture arena, the research examines two canonical co-navigation scenarios—orthogonal crossing and shoulder passing—using a PR2 mobile manipulator robot. Human behavioral responses, including trajectories and head orientation, as well as subjective perceptions such as social awareness and interaction comfort, are comprehensively evaluated for consistency between real and virtual settings. The findings demonstrate that VR accurately reproduces real-world interaction behaviors and perceptual experiences, providing the first empirical evidence supporting the ecological validity of VR as a platform for human–robot interaction (HRI) research.

Abstract

Virtual Reality (VR) offers a flexible and controllable platform for studying human-robot interaction. Prior work has explored VR for socially aware robot navigation. However, whether VR captures the multimodal interaction dynamics observed in real-world human-robot co-navigation remains insufficiently understood. In this work, we present a VR prototype and evaluate its suitability for studying multimodal human-robot interaction (HRI) in socially aware navigation. Specifically, we investigate whether VR preserves the multimodal interaction dynamics observed in real-world human-robot co-navigation. We conducted a within-subjects study (N = 21) in which participants interacted with a PR2 mobile manipulator robot in both a motion capture equipped arena and its virtual replica in an immersive VR environment. Two common co-navigation scenarios were examined : orthogonal crossing and pass-by interactions. Participants evaluated the robot's perceived social awareness and interaction comfort, while trajectory and head-orientation data were analysed to examine behavioral responses during the interaction. Our results show that participants perceive the robot's socially aware navigation similarly in VR and in the real world. Furthermore, VR captures human interaction behaviors in ways consistent with real-world observations. These findings suggest that VR can be a reliable and flexible platform for studying richer multimodal behaviors in social navigation and HRI.

Problem

• Virtual Reality
• Human-Robot Interaction
• Multimodal Interaction
• Socially Aware Navigation
• Co-navigation

Innovation

• Virtual Reality
• Multimodal Human-Robot Interaction
• Socially Aware Navigation
• Behavioral Validation
• Immersive Simulation