Predicting Human Disagreement for Calibrated Dynamic Facial Expression Recognition

2026-09-15

AI Summary

该研究提出了一种解决动态面部表情识别中人类标注分歧的方法，通过使用Dirichlet-Multinomial似然直接训练原始标注计数向量，并结合预测模糊性等进行选择性预测。

Abstract

Dynamic facial expression recognition (DFER) benchmarks such as DFEW provide multiple annotator votes per clip, yet most models collapse them to a majority label and cannot represent human disagreement at inference time. We propose a disagreement-aware DFER framework that trains directly on the raw annotator count vector using a Dirichlet-Multinomial likelihood. Unlike mean-only soft-label objectives, the proposed likelihood provides scale-sensitive supervision for the Dirichlet concentration while preserving the predictive mean. A separate ambiguity head predicts annotation entropy for unseen clips, and a monotone Chow-style reject rule combines predicted ambiguity, vacuity, temporal instability, and input quality for selective prediction. On DFEW, the method preserves recognition accuracy while reducing ECE by 30% and AURC by 15%, and predicted ambiguity reaches a Spearman correlation of 0.52 with the annotation entropy of test clips. The calibration and selective-prediction gains transfer to FERV39k and remain under identity- and movie-disjoint DFEW splits.

Problem

• Dynamic Facial Expression Recognition
• Human Disagreement
• Calibration

Innovation

• disagreement-aware DFER
• Dirichlet-Multinomial likelihood
• annotation entropy
• selective prediction
• calibration