The Role of Consequential and Functional Sound in Human-Robot Interaction: Toward Audio Augmented Reality Interfaces

2025-11-19

AI Summary

As service robots become increasingly integrated into daily life, auditory interaction must balance functional utility and social acceptability. This study distinguishes consequential sounds (e.g., operational noise) from functional sounds (intentionally designed spatial auditory cues) and systematically evaluates the impact of spatial audio cues on human perception accuracy, task efficiency, and social experience across two collaborative tasks: sound source localization and object handover. We propose a novel “dual-functional auditory design” framework that concurrently conveys precise spatial task information and enhances perceived warmth while reducing discomfort. Experiments integrate robotic acoustic characterization, real-time spatial audio rendering, and auditory cue modeling. Results show operational noise elicited no negative perceptual responses; lateral sound source localization error remained below 10°; and functional spatial cues significantly improved handover success rate (+23.6%) and subjective willingness to collaborate (p < 0.01). This work establishes theoretical foundations and design principles for empathetic, audio-augmented human–robot collaboration interfaces.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

As robots become increasingly integrated into everyday environments, understanding how they communicate with humans is critical. Sound offers a powerful channel for interaction, encompassing both operational noises and intentionally designed auditory cues. In this study, we examined the effects of consequential and functional sounds on human perception and behavior, including a novel exploration of spatial sound through localization and handover tasks. Results show that consequential sounds of the Kinova Gen3 manipulator did not negatively affect perceptions, spatial localization is highly accurate for lateral cues but declines for frontal cues, and spatial sounds can simultaneously convey task-relevant information while promoting warmth and reducing discomfort. These findings highlight the potential of functional and transformative auditory design to enhance human-robot collaboration and inform future sound-based interaction strategies.

Problem

• Examining how consequential and functional sounds affect human perception in robot interactions
• Investigating spatial sound accuracy through localization and handover tasks with robots
• Exploring auditory design to enhance collaboration and reduce discomfort in human-robot interfaces

Innovation

• Using spatial sound for robot communication
• Combining task information with emotional cues
• Applying audio augmented reality in robotics