Interpretable Uncertainty Routing Separating Emotion Ambiguity from Distribution Shift in Facial Expression Recognition

2026-06-21

AI Summary

This work addresses the challenge in facial expression recognition where uncertainty arising from emotional ambiguity and distribution shift becomes entangled, hindering models from distinguishing between ambiguous in-distribution samples to retain and out-of-distribution inputs to reject. To resolve this, the authors propose an Uncertainty-Aware Routing (UAR) mechanism that disentangles uncertainty during inference into aleatoric (reflecting emotional ambiguity) and epistemic (indicating distribution shift) components, enabling differentiated decision-making. Using a novel dual-validation protocol, the study is the first to explicitly separate and validate these two uncertainty types. Built upon a fully fine-tuned DINOv2 deep ensemble, UAR achieves interpretable routing: aleatoric uncertainty correlates with human annotation disagreement (Spearman’s ρ = 0.66), while epistemic uncertainty yields an AUROC of 0.699 under severe image corruption. At equal rejection rates, UAR retains approximately 1.8× more ambiguous samples than single-uncertainty approaches.

Abstract

Facial expression recognition (FER) is inherently ambiguous: human annotators frequently disagree, and models deployed in real environments face distribution shift. Crucially, these two conditions demand different downstream actions, as ambiguous in-distribution faces should be reported with their ambiguity whereas out-of-distribution inputs should be rejected. However, a single uncertainty score conflates the two. In this study, uncertainty decomposition into aleatoric and epistemic components for FER is investigated, and Uncertainty-Aware Routing (UAR), an inference-time routing mechanism that exploits the separation, is introduced. Specifically, aleatoric and epistemic uncertainties are obtained from a Deep Ensemble of fully fine-tuned DINOv2 models and are each validated against an independent external signal: aleatoric against human annotator disagreement, and epistemic against distribution shift induced by image corruptions. The proposed dual-validation protocol reveals that aleatoric recovers annotator disagreement with Spearman correlation 0.66 (95% CI: 0.64-0.68), and epistemic detects corruption-induced shifts, achieving average AUROC of 0.699 at the highest corruption severity. UAR retains approximately 1.8 times more ambiguous in-distribution faces than single-uncertainty routing at a matched out-of-distribution rejection rate. A strong label-distribution-learning baseline achieves comparable disagreement recovery but cannot separate ambiguity from shift and therefore cannot route, establishing that the value of decomposition lies in the separation enabling interpretable and differentiated action selection.

Problem

• facial expression recognition
• uncertainty decomposition
• emotion ambiguity
• distribution shift
• aleatoric uncertainty

Innovation

• Uncertainty Decomposition
• Aleatoric Uncertainty
• Epistemic Uncertainty
• Uncertainty-Aware Routing
• Facial Expression Recognition