Navigating Label Ambiguity for Facial Expression Recognition in the Wild

2025-02-14

AI Summary

This work addresses the dual challenges of label ambiguity and class imbalance in real-world facial expression recognition (FER). We propose the first end-to-end training framework that dynamically perceives and handles ambiguous samples. Methodologically, we jointly model label uncertainty and class distribution shift—introducing a Noise-Aware Adaptive Weighting (NAW) mechanism that integrates intermediate prediction correlation modeling and latent-space consistency regularization, complemented by an iterative hard-sample focusing strategy. Extensive experiments on challenging in-the-wild benchmarks—including RAF-DB and AffectNet—demonstrate substantial improvements in overall accuracy and mean average precision (mAP), particularly for minority classes. Our approach achieves superior robustness compared to current state-of-the-art methods, validating its effectiveness in mitigating both ambiguity-induced noise and imbalance-induced bias in practical FER settings.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Facial expression recognition (FER) remains a challenging task due to label ambiguity caused by the subjective nature of facial expressions and noisy samples. Additionally, class imbalance, which is common in real-world datasets, further complicates FER. Although many studies have shown impressive improvements, they typically address only one of these issues, leading to suboptimal results. To tackle both challenges simultaneously, we propose a novel framework called Navigating Label Ambiguity (NLA), which is robust under real-world conditions. The motivation behind NLA is that dynamically estimating and emphasizing ambiguous samples at each iteration helps mitigate noise and class imbalance by reducing the model's bias toward majority classes. To achieve this, NLA consists of two main components: Noise-aware Adaptive Weighting (NAW) and consistency regularization. Specifically, NAW adaptively assigns higher importance to ambiguous samples and lower importance to noisy ones, based on the correlation between the intermediate prediction scores for the ground truth and the nearest negative. Moreover, we incorporate a regularization term to ensure consistent latent distributions. Consequently, NLA enables the model to progressively focus on more challenging ambiguous samples, which primarily belong to the minority class, in the later stages of training. Extensive experiments demonstrate that NLA outperforms existing methods in both overall and mean accuracy, confirming its robustness against noise and class imbalance. To the best of our knowledge, this is the first framework to address both problems simultaneously.

Problem

• Addresses label ambiguity in facial expression recognition
• Mitigates class imbalance in real-world datasets
• Proposes a novel framework to handle both challenges

Innovation

• Noise-aware Adaptive Weighting
• consistency regularization
• dynamically estimating ambiguous samples