Leveraging Cognitive States for Adaptive Scaffolding of Understanding in Explanatory Tasks in HRI

2025-03-25

AI Summary

In human-AI explanatory dialogue, inconsistent user comprehension hinders effective communication. Method: We propose an adaptive linguistic scaffolding strategy—incorporating negation and hesitation markers—guided by real-time cognitive state estimation. The SHIFT computational model integrates eye-tracking data, task performance metrics, and interaction history to jointly model cognitive states and dynamically select scaffolding strategies in a closed-loop manner. A five-dimensional cognitive state scoring mechanism and a dedicated explanatory dialogue framework enable fine-grained, context-sensitive strategy modulation. Contribution/Results: Experiments demonstrate that the adaptive scaffolding reduces overall error rate by 22.8% compared to a fixed affirmative strategy; significant improvements in comprehension are observed across three of five cognitive states, validating both contextual adaptability and empirical efficacy. This work establishes the first integrated framework linking real-time cognitive modeling with dynamic linguistic scaffolding in explanatory AI dialogue.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Understanding how scaffolding strategies influence human understanding in human-robot interaction is important for developing effective assistive systems. This empirical study investigates linguistic scaffolding strategies based on negation as an important means that de-biases the user from potential errors but increases processing costs and hesitations as a means to ameliorate processing costs. In an adaptive strategy, the user state with respect to the current state of understanding and processing capacity was estimated via a scoring scheme based on task performance, prior scaffolding strategy, and current eye gaze behavior. In the study, the adaptive strategy of providing negations and hesitations was compared with a non-adaptive strategy of providing only affirmations. The adaptive scaffolding strategy was generated using the computational model SHIFT. Our findings indicate that using adaptive scaffolding strategies with SHIFT tends to (1) increased processing costs, as reflected in longer reaction times, but (2) improved task understanding, evidenced by a lower error rate of almost 23%. We assessed the efficiency of SHIFT's selected scaffolding strategies across different cognitive states, finding that in three out of five states, the error rate was lower compared to the baseline condition. We discuss how these results align with the assumptions of the SHIFT model and highlight areas for refinement. Moreover, we demonstrate how scaffolding strategies, such as negation and hesitation, contribute to more effective human-robot explanatory dialogues.

Problem

• Investigating how scaffolding strategies affect human understanding in HRI
• Adaptive scaffolding based on user state improves task performance
• Comparing negation and hesitation strategies with affirmations in HRI dialogues

Innovation

• Adaptive scaffolding based on cognitive states
• SHIFT model generates negation and hesitation strategies
• Eye gaze behavior estimates user understanding