AU-Guided Synthetic Video Generation for Micro-Expression Recognition

2026-07-12

AI Summary

Current micro-expression datasets suffer from limited scale, narrow demographic coverage, and simplistic emotion labels, severely hindering model generalization. To address these limitations, this work proposes an action unit (AU)-guided image-to-video generation framework to construct EquiME, a large-scale synthetic micro-expression dataset comprising 75K video samples, each automatically annotated with demographic attributes and quality metrics. By leveraging structured AU-based conditioning, the method enables the first controllable, diverse, and metadata-rich synthesis of micro-expressions. Models trained on EquiME demonstrate strong cross-dataset generalization on SAMM and CASME II benchmarks and exhibit consistent performance across multiple architectures.

Abstract

Micro-expression recognition is limited by the small scale, narrow demographic coverage, and restricted emotion labels of existing datasets. We introduce EquiME, a synthetic micro-expression dataset built from AU-guided image-to-video generation. EquiME contains 75K videos generated from 15K source face images across five target emotions, together with automatically inferred demographic metadata and video-quality measurements. We evaluate EquiME using frame-pair similarity, spatial variation, and no-reference perceptual-quality metrics, together with cross-dataset MER experiments on SAMM and CASME II. Models trained on EquiME achieve competitive cross-dataset performance on SAMM and CASME II and show comparatively low variation across the four evaluated architectures. This paper focuses on the dataset design, the structured AU-conditioning pipeline used for video generation, and the empirical evidence needed to assess EquiME as a synthetic MER resource. Project page: https://kirito-blade.github.io/me-vlm/

Problem

• micro-expression recognition
• dataset limitation
• emotion labels
• demographic coverage
• synthetic data

Innovation

• AU-guided generation
• synthetic micro-expression dataset
• image-to-video synthesis
• cross-dataset evaluation
• micro-expression recognition