Predicting Cognitive Load from Speech and Interaction Dynamics in Dyadic Conversations

2026-06-11

AI Summary

Existing approaches struggle to reliably estimate cognitive load in natural collaborative dialogue, often being confined to controlled laboratory settings. This study leverages audio recordings from 53 participant pairs engaged in nine collaborative tasks and integrates static acoustic features with dynamic and interactive cues—such as turn-taking overlap and speaking imbalance—to model multidimensional cognitive load. We propose a dual-head Gated Recurrent Unit (GRU) encoder that, for the first time, reveals robust associations between interactional dynamics in natural conversation and dimensions of cognitive load, including time pressure and mental demand. Results demonstrate that dialogic interaction signals effectively predict overall cognitive load and its key components, while also exhibiting significant links to specific interaction patterns, thereby underscoring the critical role of task structure and conversational dynamics in cognitive load modeling.

Abstract

Estimating cognitive load from speech has largely been studied in controlled laboratory settings, with limited understanding of its reliability in natural collaborative conversations. We investigate whether speech and interaction dynamics predict perceived cognitive load during dyadic conversations. We analyze audio from 53 dyads performing nine collaborative tasks and extract static acoustic, dynamic, and interaction features to train a two-head Gated Recurrent Unit encoder to predict cognitive load scores. Results show conversational interaction provides useful signals for predicting cognitive load related to time pressure, mental work, effort, and task performance. Temporal demand is associated with turn-taking dynamics such as overlap and speaker switch, while mental demand is linked to imbalanced participation between speakers. These findings highlight the importance of task structure and conversational interaction for modeling cognitive load in natural collaborative settings.

Problem

• cognitive load
• dyadic conversations
• speech
• interaction dynamics
• collaborative tasks

Innovation

• cognitive load prediction
• dyadic conversation
• interaction dynamics
• speech features
• GRU encoder