Using Physiological Measures, Gaze, and Facial Expressions to Model Human Trust in a Robot Partner

2025-04-07

AI Summary

Traditional human–robot trust modeling relies heavily on subjective self-report scales, lacking objective, real-time measurement capabilities in human–robot collaborative supervision scenarios. Method: We propose a novel co-located supervisory interaction paradigm to collect multimodal physiological and behavioral data—including photoplethysmography (PPG), electrodermal activity (EDA), skin temperature, eye-tracking, and RGB facial video—yielding the first dedicated multimodal dataset for face-to-face human–robot collaboration trust assessment. We further design a cross-modal fusion framework for trust state recognition, employing tree-based models (e.g., Extra Trees) for cross-subject classification. Contribution/Results: Experimental results demonstrate that multimodal fusion significantly improves classification accuracy over unimodal baselines. Extra Trees achieves superior generalizability and real-time inference capability, enabling online trust modeling and adaptive human–robot interaction. This work establishes both a foundational dataset and a robust methodological framework for objective, dynamic trust quantification in collaborative settings.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

With robots becoming increasingly prevalent in various domains, it has become crucial to equip them with tools to achieve greater fluency in interactions with humans. One of the promising areas for further exploration lies in human trust. A real-time, objective model of human trust could be used to maximize productivity, preserve safety, and mitigate failure. In this work, we attempt to use physiological measures, gaze, and facial expressions to model human trust in a robot partner. We are the first to design an in-person, human-robot supervisory interaction study to create a dedicated trust dataset. Using this dataset, we train machine learning algorithms to identify the objective measures that are most indicative of trust in a robot partner, advancing trust prediction in human-robot interactions. Our findings indicate that a combination of sensor modalities (blood volume pulse, electrodermal activity, skin temperature, and gaze) can enhance the accuracy of detecting human trust in a robot partner. Furthermore, the Extra Trees, Random Forest, and Decision Trees classifiers exhibit consistently better performance in measuring the person's trust in the robot partner. These results lay the groundwork for constructing a real-time trust model for human-robot interaction, which could foster more efficient interactions between humans and robots.

Problem

• Model human trust in robots using physiological and behavioral data
• Develop real-time trust prediction for human-robot interactions
• Identify optimal sensor modalities for accurate trust detection

Innovation

• Combining physiological measures for trust modeling
• Machine learning for trust prediction accuracy
• Real-time trust model enhances human-robot interaction