IM HERE: Interaction Model for Human Effort Based Robot Engagement

2025-06-02

Conference on Cognitive and Computational Aspects of Situation Management

AI Summary

Existing engagement models are typically vague and lack generalizability, hindering natural human–machine–social interaction. This paper proposes the first unified dynamic engagement modeling framework that jointly encodes subjective perception and objective behavioral states, supporting human–human, human–robot, and robot–robot interaction scenarios. Centered on “cognitive effort,” the framework quantifies attention allocation, identifies four fundamental engagement states, and enables automatic misalignment detection and socially norm-constrained behavior generation. By integrating social behavior analysis with autonomous system control techniques, our approach achieves, for the first time, cross-agent, interpretable, and intervenable engagement state modeling. Experimental results demonstrate significant improvements in robots’ social goal achievement and interaction naturalness. The framework establishes both theoretical foundations and technical infrastructure for the societal integration of embodied intelligence.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

The effectiveness of human-robot interaction often hinges on the ability to cultivate engagement—a dynamic process of cognitive involvement that supports meaningful exchanges. Many existing definitions and models of engagement are either too vague or lack the ability to generalize across different contexts. We introduce IM HERE, a novel framework that models engagement effectively in human-human, human-robot, and robot-robot interactions. By employing an effort-based description of bilateral relationships between entities, we provide an accurate breakdown of relationship patterns, simplifying them to focus placement and four key states. This framework captures mutual relationships, group behaviors, and actions conforming to social norms, translating them into specific directives for autonomous systems. By integrating both subjective perceptions and objective states, the model precisely identifies and describes miscommunication. The primary objective of this paper is to automate the analysis, modeling, and description of social behavior, and to determine how autonomous systems can behave in accordance with social norms for full social integration while simultaneously pursuing their own social goals.

Problem

• Models engagement in human-robot interactions using effort-based relationships
• Identifies and describes miscommunication by integrating subjective and objective states
• Automates social behavior analysis for autonomous systems to follow norms

Innovation

• Effort-based bilateral relationship modeling for engagement
• Four-state simplification of relationship patterns and focus placement
• Integration of subjective perceptions with objective states for miscommunication identification