SRE-FER: Regional residual evidence learning for mitigating local evidence dilution in fine-grained facial expression recognition

2026-08-09

AI Summary

This work addresses the challenge in fine-grained facial expression recognition where global feature aggregation often dilutes subtle local muscle cues, leading to confusion between adjacent emotion categories such as fear/surprise and sadness/neutral. To mitigate this, the authors propose the SRE-FER framework, which introduces region-wise residual evidence learning at the readout layer. By employing zero-initialized residual logits, the method sharpens class boundaries while preserving the backbone’s global predictions. A Region-Enhanced Residual Attention (RERA) module leverages Facial Action Coding System (FACS) anatomical priors to guide regional feature focus toward expression-relevant areas, eliminating the need for external keypoints during inference. An optional Full setting dynamically selects non-redundant tokens for efficiency. Built upon a DINOv3 backbone, the approach achieves state-of-the-art accuracy of 92.76%, 91.32%, and 67.78% on RAF-DB, FERPlus, and AffectNet-7, respectively.

Abstract

Fine-grained facial expression recognition (FER) hinges on capturing subtle muscular cues that distinguish adjacent emotions. Yet capturing these cues presents a dilemma. Detector-based methods depend on fragile landmark pipelines, whereas we find that directly transferring foundation models such as DINOv3 under conventional global readouts can cause local evidence dilution: early global aggregation washes out sparse muscular signals and leaves persistent confusion between categories such as fear/surprise and sad/neutral. To recover this evidence, we propose SRE-FER, a readout-level regional residual evidence learning framework. Its core module, RERA, adds zero-initialized residual logits that refine class boundaries while preserving the backbone's global prediction. Training-time action unit (AU) guidance steers regional features toward expression-relevant areas using Facial Action Coding System (FACS)-based anatomical priors, without requiring an external facial pipeline at inference. An optional Full setting further routes sample-specific non-redundant tokens. On three benchmarks, SRE-FER attains 92.76% on RAF-DB, 91.32% on FERPlus, and 67.78% on AffectNet-7, demonstrating highly competitive performance compared to existing FER methods.

Problem

• fine-grained facial expression recognition
• local evidence dilution
• global aggregation
• subtle muscular cues
• emotion confusion

Innovation

• regional residual evidence learning
• local evidence dilution
• action unit guidance
• FACS-based anatomical priors
• readout-level refinement