REVELIO -- Universal Multimodal Task Load Estimation for Cross-Domain Generalization

2025-09-01

AI Summary

Existing cognitive load estimation models suffer from poor generalizability, with insufficient evaluation of robustness and transferability in real-world settings. To address this, we propose the first cross-domain transferable cognitive load assessment framework. We introduce a novel, ecologically valid multimodal dataset integrating n-back tasks and game-based applications, annotated along three dimensions: objective behavioral performance, subjective self-reports, and task design parameters. Leveraging end-to-end architectures—including xLSTM, ConvNeXt, and Transformer—we jointly model heterogeneous physiological and behavioral signals. A systematic cross-domain evaluation reveals that multimodal approaches significantly outperform unimodal baselines; however, non-negligible performance degradation persists under domain shifts, exposing fundamental challenges in generalizable cognitive load estimation. This work establishes a reproducible, extensible cross-domain benchmark, providing both a new paradigm for transferable cognitive modeling and a strong, publicly available baseline.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Task load detection is essential for optimizing human performance across diverse applications, yet current models often lack generalizability beyond narrow experimental domains. While prior research has focused on individual tasks and limited modalities, there remains a gap in evaluating model robustness and transferability in real-world scenarios. This paper addresses these limitations by introducing a new multimodal dataset that extends established cognitive load detection benchmarks with a real-world gaming application, using the $n$-back test as a scientific foundation. Task load annotations are derived from objective performance, subjective NASA-TLX ratings, and task-level design, enabling a comprehensive evaluation framework. State-of-the-art end-to-end model, including xLSTM, ConvNeXt, and Transformer architectures are systematically trained and evaluated on multiple modalities and application domains to assess their predictive performance and cross-domain generalization. Results demonstrate that multimodal approaches consistently outperform unimodal baselines, with specific modalities and model architectures showing varying impact depending on the application subset. Importantly, models trained on one domain exhibit reduced performance when transferred to novel applications, underscoring remaining challenges for universal cognitive load estimation. These findings provide robust baselines and actionable insights for developing more generalizable cognitive load detection systems, advancing both research and practical implementation in human-computer interaction and adaptive systems.

Problem

• Develops universal task load estimation model for cross-domain generalization
• Evaluates multimodal model robustness in real-world gaming scenarios
• Addresses performance degradation when transferring models to novel applications

Innovation

• Multimodal dataset with gaming application
• xLSTM ConvNeXt Transformer architectures evaluated
• Cross-domain generalization performance assessed