AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow

2026-07-14

AI Summary

This work addresses the ambiguity of facial behavior in unconstrained settings by proposing a conditional rectified flow–based multitask emotion estimation framework. Building upon a frozen DINOv2 ViT-S/16 backbone, the method introduces a conditional rectified flow head that models conditional generative distributions to enable uncertainty-aware one-to-many predictions, applied for the first time to the joint estimation of valence-arousal dimensions, facial expressions, and action units. Coupled with post-hoc threshold calibration, the approach substantially improves performance on rare classes without requiring retraining. Evaluated on the ABAW challenge, the method achieves a P_MTL score of 1.177—surpassing the baseline by 0.45—with a 0.058 gain in valence concordance correlation coefficient and a dramatic increase in fear expression recognition rate from 3.8% to 33.1%.

Abstract

We present \textbf{AffectFlow-DINO}, a multi-task learning system for the 11th ABAW challenge that extends a standard deterministic architecture with a conditional rectified-flow head to model the inherent ambiguity of in-the-wild facial behavior. Instead of predicting a single affect estimate, the model learns a conditional generative distribution, enabling uncertainty-aware one-to-many predictions through Monte Carlo sampling. The system jointly estimates continuous valence-arousal, classifies eight facial expressions, and detects twelve Action Units from static face images. Built on a frozen DINOv3 ViT-S/16 backbone, extensive ablation studies show that rectified-flow decoding consistently improves deterministic prediction, particularly for valence-arousal estimation (CCC-V $+0.058$). We further show that post-hoc threshold calibration effectively recovers performance on severely imbalanced rare classes (e.g., Fear: $3.8\% \rightarrow 33.1\%$) without retraining. Combined with backbone fine-tuning and flow retuning, the final model achieves $\mathbf{P_{MTL}=1.177}$, substantially outperforming the official challenge baseline of $P_{MTL}=0.45$.

Problem

• affect estimation
• multi-task learning
• facial behavior ambiguity
• uncertainty-aware prediction
• in-the-wild

Innovation

• conditional rectified flow
• uncertainty-aware prediction
• multi-task affect estimation
• post-hoc threshold calibration
• DINOv3 backbone