🤖 AI Summary
This study addresses the poor performance of in-the-wild facial expression recognition on rare emotion categories, arguing that the issue stems not merely from data imbalance but from representational degeneration within the circular affective space defined by Russell’s circumplex model. The authors propose a novel approach grounded in optimal transport theory to construct an emotion-distance-based cost function, integrated with action unit augmentation and uniform cost regularization. Multi-task ablation experiments on AffectNet and Aff-Wild2 reveal that the core bottleneck lies in the geometric degradation of the affective space. The work further introduces a new evaluation paradigm distinguishing representational capacity from confusion-aware reweighting. Results show that while the proposed circular cost improves standard metrics, uniform costing yields superior performance on AffectNet; predictions on Aff-Wild2 better reflect true affective distributions, yet fine-grained distinctions—such as between anger and fear—remain challenging.
📝 Abstract
In-the-wild expression recognition persistently fails on a few rare emotions, and the standard explanation is class imbalance. Through a controlled multi-task study on two benchmarks, we show the failure is instead a property of affect geometry: the rare classes are degenerate on Russell's circumplex, and that degeneracy bounds what any loss or cost can achieve. Our instrument is a circumplex-cost optimal-transport term that prices expression confusions by their valence-arousal distance. The term improves the official score and expression macro-F1, but a control most studies omit shows the gain is not geometric: a uniform cost, equivalent to a generic confidence penalty, matches it on Aff-Wild2 (p=0.625) and significantly exceeds it on AffectNet (+0.057 over base, larger than the circumplex). What the geometry reshapes is the structure of the errors, making them affectively nearer the truth on Aff-Wild2 (p=0.031 against the uniform control), an effect that does not survive on AffectNet, where a visual confound at the far corner of the circumplex overwhelms it. The rare-class failure, by contrast, is stable across both datasets we examine: the degenerate pairs (anger-fear on Aff-Wild2, anger-contempt on AffectNet) resist frequency-based interventions, the transport term, and an action-unit-augmented cost built specifically to separate them. We conclude that progress on rare expressions requires representations that distinguish the classes, not supervision that reprices their confusions, and we provide the controls and metrics needed to tell the two apart.