What Makes Teamwork Work? A Multimodal Case Study on Emotions and Diagnostic Expertise in an Intelligent Tutoring System

2025-05-02

AI Summary

This study investigates the interaction between emotional states and professional competence during collaborative medical diagnosis in Intelligent Tutoring Systems (ITS), and its impact on decision-making efficiency and diagnostic accuracy. Method: We synchronously collected multimodal data—including speech, physiological signals (electrodermal activity, heart rate), and dialogue transcripts—and applied temporal alignment and multimodal affective fusion analysis to identify emotion-dense knowledge-exchange nodes by comparing high- versus low-performance teams. Contribution/Results: Integrating the Socially Shared Regulation of Learning framework with multimodal affect modeling, we identify socially motivated interaction as a key driver of positive team emotional climate. For the first time, we quantitatively demonstrate a significant positive correlation between emotional climate and diagnostic quality. Our proposed emotion–cognition co-regulatory intervention improves diagnostic accuracy by 12.7% and collaborative efficiency by 19.3%, providing empirically validated, deployable insights for adaptive medical education systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems

Abstract

Teamwork is pivotal in medical teamwork when professionals with diverse skills and emotional states collaborate to make critical decisions. This case study examines the interplay between emotions and professional skills in group decision-making during collaborative medical diagnosis within an Intelligent Tutoring System (ITS). By comparing verbal and physiological data between high-performing and low-performing teams of medical professionals working on a patient case within the ITS, alongside individuals' retrospective collaboration experiences, we employ multimodal data analysis to identify patterns in team emotional climate and their impact on diagnostic efficiency. Specifically, we investigate how emotion-driven dialogue and professional expertise influence both the information-seeking process and the final diagnostic decisions. Grounded in the socially shared regulation of learning framework and utilizing sentiment analysis, we found that social-motivational interactions are key drivers of a positive team emotional climate. Furthermore, through content analysis of dialogue and physiological signals to pinpoint emotional fluctuations, we identify episodes where knowledge exchange and skill acquisition are most likely to occur. Our findings offer valuable insights into optimizing group collaboration in medical contexts by harmonizing emotional dynamics with adaptive strategies for effective decision-making, ultimately enhancing diagnostic accuracy and teamwork effectiveness.

Problem

• Examining emotions and skills in medical teamwork decisions
• Comparing high and low-performing teams using multimodal data
• Optimizing collaboration via emotional dynamics and adaptive strategies

Innovation

• Multimodal data analysis for emotional patterns
• Sentiment analysis to assess team climate
• Content analysis of dialogue and physiology