STAG: Spatio-temporal Evolving Structural Representation of Action Units for Micro-expression Recognition

2026-06-26

AI Summary

Micro-expression recognition remains highly challenging due to subtle facial movements and extremely short durations. Existing approaches often rely on apex-onset frame pairs, neglect inter-frame dynamics, and treat spatial and temporal modeling separately, limiting their generalization capability. To address these issues, this work proposes the STAG framework, which introduces an AU-guided dynamic ROI linking mechanism to adaptively model facial muscle activation patterns. It integrates magnitude-aware optical flow with temporal attention, enhances graph attention networks with a Transformer encoder, and employs bidirectional cross-attention to mutually refine spatial and temporal features. Extensive experiments on multiple benchmarks—including CASME II, 4DME, DFME, NaME, SAMM, and SMIC-HS—demonstrate that STAG significantly improves robustness, generalization, interpretability, and computational efficiency in micro-expression recognition.

Abstract

Micro-expression recognition is challenging due to subtle and short-lived facial muscle movements. Existing methods rely heavily on apex-onset frames, overlook fine-grained inter-frame dynamics, and separately model spatial and temporal information, limiting generalization across datasets. To address these challenges, we propose STAG, a dynamic ROI-AU-coupled spatial-temporal network that jointly models motion flow and adaptive facial connectivity. The framework extracts optical flow from discriminative frames using magnitude-based selection and temporal attention. A dual-branch architecture combines an enhanced graph attention network for structured spatial reasoning with a transformer encoder for temporal modeling. A bidirectional cross-attention module enables mutual refinement of spatial and temporal features, while AU-guided dynamic connectivity adapts facial region interactions according to muscle activation patterns. The transformer captures subtle temporal dynamics beyond apex-based approaches, improving semantic consistency and interpretability for explainable micro-expression recognition. The fused representation is optimized using focal loss and evaluated on CASME II, 4DME, DFME, NaME, SAMM, and SMIC-HS. Extensive experiments demonstrate improved robustness, generalization, interpretability, and computational efficiency, confirming the effectiveness of adaptive relational reasoning, AU-guided dynamic connectivity, and deep spatial-temporal feature fusion for accurate cross-dataset micro-expression recognition.

Problem

• micro-expression recognition
• spatio-temporal modeling
• fine-grained dynamics
• cross-dataset generalization
• facial muscle movements

Innovation

• micro-expression recognition
• spatio-temporal modeling
• action unit-guided connectivity
• graph attention network
• transformer-based temporal dynamics