Emotions as Ambiguity-aware Ordinal Representations

2025-08-26

AI Summary

Existing continuous emotion recognition methods often overlook the inherent ambiguity of emotions or model it as static, independent variables, failing to capture its temporal dynamic nature. This work proposes a novel fuzzy ordinal emotion modeling framework that explicitly represents emotional ambiguity as time-evolving ordinal relationships—quantified via temporal change rates—to jointly model both bounded and unbounded emotion trajectories. Integrating ordinal regression with ambiguity-aware mechanisms, the framework captures relative emotional shifts and uncertainty dynamics in a unified manner. Evaluated on RECOLA and GameVibe, it achieves state-of-the-art CCC and SDA scores under unbounded annotations, and yields significant SDA improvements under bounded annotations. The approach advances continuous emotion modeling by explicitly encoding ambiguity’s temporal evolution, thereby enhancing robustness to label uncertainty and improving representation of affective dynamics.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Humans and AI: Emotional Intelligence

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Emotions are inherently ambiguous and dynamic phenomena, yet existing continuous emotion recognition approaches either ignore their ambiguity or treat ambiguity as an independent and static variable over time. Motivated by this gap in the literature, in this paper we introduce emph{ambiguity-aware ordinal} emotion representations, a novel framework that captures both the ambiguity present in emotion annotation and the inherent temporal dynamics of emotional traces. Specifically, we propose approaches that model emotion ambiguity through its rate of change. We evaluate our framework on two affective corpora -- RECOLA and GameVibe -- testing our proposed approaches on both bounded (arousal, valence) and unbounded (engagement) continuous traces. Our results demonstrate that ordinal representations outperform conventional ambiguity-aware models on unbounded labels, achieving the highest Concordance Correlation Coefficient (CCC) and Signed Differential Agreement (SDA) scores, highlighting their effectiveness in modeling the traces' dynamics. For bounded traces, ordinal representations excel in SDA, revealing their superior ability to capture relative changes of annotated emotion traces.

Problem

• Modeling ambiguity and dynamics in continuous emotion recognition
• Addressing limitations of existing ambiguity-aware emotion representations
• Evaluating ordinal representations on bounded and unbounded affective traces

Innovation

• Ambiguity-aware ordinal emotion representations
• Modeling emotion ambiguity through rate of change
• Ordinal representations outperform conventional ambiguity-aware models