Inter-Stance: A Dyadic Multimodal Corpus for Conversational Stance Analysis

2026-04-24

AI Summary

This study addresses the scarcity of publicly available datasets that simultaneously capture multimodal behavioral signals and self-reported annotations, which has hindered the modeling of stance (agree/disagree/neutral) and social cues in dyadic conversations. To bridge this gap, the authors present a novel multimodal interaction corpus comprising 45 dyads (90 participants), encompassing both familiar and unfamiliar pairs. The dataset integrates synchronized recordings of 2D/3D facial video, thermal imaging, audio, and multiple physiological signals—including photoplethysmography (PPG), electrodermal activity (EDA), heart rate, blood pressure, and respiration—alongside structured stance labels and subjective emotional reports. Spanning approximately 20 terabytes, this resource is the first to jointly provide fine-grained behavioral data and introspective self-assessments, thereby enabling robust computational modeling of interpersonal social dynamics and filling a critical void in the field.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Interaction Techniques and Devices

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements

Abstract

Social interactions dominate our perceptions of the world and shape our daily behavior by attaching social meaning to acts as simple and spontaneous as gestures, facial expressions, voice, and speech. People mimic and otherwise respond to each other's postures, facial expressions, mannerisms, and other verbal and nonverbal behavior, and form appraisals or evaluations in the process. Yet, no publicly-available dataset includes multimodal recordings and self-report measures of multiple persons in social interaction. Dyadic recordings and annotation are lacking. We present a new data corpus of multimodal dyadic interaction (45 dyads, 90 persons) that includes synchronized multi-modality behavior (2D face video, 3D face geometry, thermal spectrum dynamics, voice and speech behavior, physiology (PPG, EDA, heart-rate, blood pressure, and respiration), and self-reported affect of all participants in a communicative interaction scenario. Two types of dyads are included: persons with shared past history and strangers. Annotations include social signals, agreement, disagreement, and neutral stance. With a potent emotion induction, these multimodal data will enable novel modeling of multimodal interpersonal behavior. We present extensive experiments to evaluate multimodal dyadic communication of dyads with and without interpersonal history, and their affect. This new database will make multimodal modeling of social interaction never possible before. The dataset includes 20TB of multimodal data to share with the research community.

Problem

• dyadic interaction
• multimodal corpus
• conversational stance
• social signals
• affect annotation

Innovation

• multimodal dyadic interaction
• conversational stance analysis
• synchronized physiological signals
• social signal annotation
• interpersonal history