Are We Generalizing from the Exception? An In-the-Wild Study on Group-Sensitive Conversation Design in Human-Agent Interactions

2025-06-12

AI Summary

Group-sensitive dialogue design remains underexplored in multi-user human-agent interaction, particularly regarding how embodiment modality—physical robots versus virtual agents—affects adaptation to dynamic group states. Method: We conducted an in-situ study at a German museum with 188 participants organized in 2–8-person groups, deploying a hybrid retrieval-generation AI backend that integrates speech, behavioral cues, and context-aware group-state modeling for multimodal dialogue adaptation. We empirically compared two embodied agents: the physical Furhat robot and the virtual MetaHuman agent. Contribution/Results: Results show that linguistic pluralization alone is insufficient for effective group interaction; robust adaptation requires tight cross-modal coordination among speech, vision, and behavior. Strong coupling exists among group size, embodiment modality, and interaction modality. Although no statistically significant improvement in user satisfaction was observed, this work provides foundational empirical evidence and actionable design principles for multi-agent human-AI and human-robot interaction (HAI/HRI).

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Interaction Techniques and Devices
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

This paper investigates the impact of a group-adaptive conversation design in two socially interactive agents (SIAs) through two real-world studies. Both SIAs - Furhat, a social robot, and MetaHuman, a virtual agent - were equipped with a conversational artificial intelligence (CAI) backend combining hybrid retrieval and generative models. The studies were carried out in an in-the-wild setting with a total of $N = 188$ participants who interacted with the SIAs - in dyads, triads or larger groups - at a German museum. Although the results did not reveal a significant effect of the group-sensitive conversation design on perceived satisfaction, the findings provide valuable insights into the challenges of adapting CAI for multi-party interactions and across different embodiments (robot vs. virtual agent), highlighting the need for multimodal strategies beyond linguistic pluralization. These insights contribute to the fields of Human-Agent Interaction (HAI), Human-Robot Interaction (HRI), and broader Human-Machine Interaction (HMI), providing insights for future research on effective dialogue adaptation in group settings.

Problem

• Investigating group-adaptive conversation design in interactive agents
• Assessing impact on satisfaction in multi-party human-agent interactions
• Exploring challenges in adapting AI for different agent embodiments

Innovation

• Hybrid retrieval and generative models
• Group-adaptive conversation design
• Multimodal strategies beyond linguistic pluralization