🤖 AI Summary
A lack of high-quality, multimodal benchmark datasets hinders progress in French affective computing. Method: This paper introduces FERG, the first French multimodal emotion dataset grounded in card-game interactions. It captures natural emotional expressions from 20 participants across 10 conversational gameplay sessions, synchronously recording facial video, speech, and hand-motion capture data. Emotions are annotated using a structured, fine-grained protocol, with cross-modal temporal alignment ensured via precise synchronization. Crucially, the dataset employs a gamified, context-aware emotion elicitation paradigm, facilitating future integration of textual (NLP) and other modalities. Contribution/Results: FERG is publicly released, comprising 10 hours of high-fidelity, time-aligned multimodal data. It fills a critical gap in French multimodal emotion resources and significantly enhances model generalizability and robustness in natural human–computer interaction scenarios.
📝 Abstract
The field of affective computing has seen significant advancements in exploring the relationship between emotions and emerging technologies. This paper presents a novel and valuable contribution to this field with the introduction of a comprehensive French multimodal dataset designed specifically for emotion recognition. The dataset encompasses three primary modalities: facial expressions, speech, and gestures, providing a holistic perspective on emotions. Moreover, the dataset has the potential to incorporate additional modalities, such as Natural Language Processing (NLP) to expand the scope of emotion recognition research. The dataset was curated through engaging participants in card game sessions, where they were prompted to express a range of emotions while responding to diverse questions. The study included 10 sessions with 20 participants (9 females and 11 males). The dataset serves as a valuable resource for furthering research in emotion recognition and provides an avenue for exploring the intricate connections between human emotions and digital technologies.