M2HRI: An LLM-Driven Multimodal Multi-Agent Framework for Personalized Human-Robot Interaction

2026-04-13

AI Summary

This work addresses a critical gap in existing multi-robot human–robot interaction systems, which commonly overlook the influence of individual robot identity on user perception and lack coordination mechanisms that balance personality differentiation with behavioral consistency. To bridge this gap, we propose the first multi-agent framework integrating large language models, multimodal perception, and long-term memory to endow each robot with a distinct personality, complemented by a centralized coordination mechanism that explicitly accounts for individual differences. A user study (n = 105) demonstrates that the designed personalities are significantly distinguishable, long-term memory effectively enhances interaction personalization, and the coordination mechanism substantially reduces behavioral redundancy while significantly improving overall human–robot interaction quality.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Multiagent Systems: Coordination and Collaboration
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Multi-robot systems hold significant promise for social environments such as homes and hospitals, yet existing multi-robot works treat robots as functionally identical, overlooking how robots individual identity shape user perception and how coordination shapes multi-robot behavior when such individuality is present. To address this, we introduce M2HRI, a multimodal multi-agent framework built on large language models that equips each robot with distinct personality and long-term memory, alongside a coordination mechanism conditioned on these differences. In a controlled user study (n = 105) in a multi-agent human-robot interaction (HRI) scenario, we find that LLM-driven personality traits are significantly distinguishable and enhance interaction quality, long-term memory improves personalization and preference awareness, and centralized coordination significantly reduces overlap while improving overall interaction quality. Together, these results demonstrate that both agent individuality and structured coordination are essential for coherent and socially appropriate multi-agent HRI. Project website and code are available at https://project-m2hri.github.io/.

Problem

• multi-robot systems
• individual identity
• human-robot interaction
• coordination
• personality

Innovation

• LLM-driven multi-agent
• robot individuality
• long-term memory
• centralized coordination
• personalized HRI