Introducing WARM-VR: Benchmark Dataset for Multimodal Wearable Affect Recognition in Virtual Reality

2026-04-30

AI Summary

This work addresses the limitation of existing affective computing datasets, which are predominantly grounded in static scenarios and thus ill-suited for immersive, multisensory environments such as virtual reality (VR). To bridge this gap, we introduce WARM-VR, the first publicly available multimodal affective benchmark dataset integrating olfactory enhancement within a VR setting. Physiological signals were synchronously recorded via wrist-worn sensors (capturing BVP, EDA, skin temperature, and acceleration) and a chest-worn ECG monitor during concurrent audiovisual-olfactory stimulation, complemented by subjective self-reports. Evaluations of CNN, CNN-Bi-GRU, and lightweight Transformer models on this dataset yield F1-scores of 0.63 for valence binary classification, with the lightweight Transformer achieving the best performance on arousal recognition and a peak average F1-score of 0.64 for relaxation-state identification, establishing a new empirical foundation for immersive affective computing.

Abstract

With the growing integration of human-computer interaction into everyday life, advances in machine learning have enabled systems to better perceive and respond to users' emotional states. Most existing affect recognition datasets focus on static environments, limiting their applicability to immersive multimedia contexts such as Virtual Reality (VR). In this paper, we introduce WARM-VR, a novel publicly available multimodal dataset designed to support affect recognition in immersive, multisensory environments using wearable sensing instrumentation. Data were collected from 31 participants aged 19-37 using wearable sensors: a wristband measuring Blood Volume Pulse (BVP), EDA, skin Temperature, three-axis Acceleration, and a chest strap recording ECG signals. Participants engaged in immersive VR experiences designed to elicit relaxation through a calming beach environment following stress induction via an arithmetic task. These sessions incorporated synchronized multimedia stimuli: visual, auditory, and olfactory. Affective states were assessed subjectively through validated self-report questionnaires and objectively through the analysis of physiological measurements. Statistical analysis of the questionnaires confirmed that VR relaxation significantly reduced negative affect, particularly with olfactory enhancement. Furthermore, we established a benchmark on the dataset using widely recognized machine learning algorithms. The best performance for binary classification from BVP data of valence, was obtained with a CNN and a CNN-Bi-GRU model, both achieving an average F1-score of 0.63 and an AUC of 0.69. For arousal, a lightweight Transformer architecture provided the most balanced results (F1-0 0.54 and F1-1 0.63), outperforming recurrent hybrids. In the relaxation task, a CNN-Bi-GRU model reached the highest overall performance (average F1-score 0.64, AUC 0.69).

Problem

• affect recognition
• virtual reality
• wearable sensors
• multimodal dataset
• immersive environments

Innovation

• multimodal affect recognition
• virtual reality
• wearable sensing
• benchmark dataset
• physiological signals