Chehre: An Emoji-Prompted Video Dataset for Perceptually Diverse Facial Expression Recognition

2026-06-19

AI Summary

Existing facial expression recognition datasets predominantly rely on static images, basic emotion categories, or single-label annotations, limiting their ability to capture the dynamics of facial expressions and the diversity of human perception. To address this, this work proposes Chehre—a novel, anonymized facial expression video dataset that leverages emojis as both expressive prompts and annotation primitives. The dataset employs facial motion transfer to generate synthetic videos for privacy preservation and collects multi-label emotion distributions via crowdsourcing. The study formulates a distributional facial expression recognition task and evaluates vision-language models under character-specific emoji prompts through multi-prediction assessment. Experiments reveal that state-of-the-art models achieve only 32.5% Top-1 accuracy on dominant emotion recognition, and their Spread Ratio on the distributional task remains substantially below human performance, highlighting the challenge and research potential of this new benchmark.

Abstract

Facial expressions are nonverbal social signals used in human interaction, but facial expression recognition datasets often focus on static images, basic emotion categories, or single deterministic annotations. We introduce Chehre, an emoji-prompted video dataset for analyzing dynamic facial expressions across a wide range of expressions for exploring inter-individual perceptual diversity. In Chehre, participants were prompted to express and record 40 facial emojis. Later, their facial motions were transferred onto synthetic faces to preserve privacy. A separate group of annotators analyzed the anonymized videos using emoji and label annotations, resulting in 2,111 high quality videos collected from 203 performers and validated by 902 annotators. We define two benchmark tasks: dominant expression recognition, which tests whether models recover the top human-rated labels, and distributional expression recognition, which tests whether models capture the diversity of human responses. We benchmark recent vision-language models using random sampling and persona prompting to generate multiple predictions per video. Results show that both tasks are challenging: among the models evaluated, the best-performing model achieves only 32.5% Top-1 accuracy on dominant expression recognition and a Spread Ratio well below the human reference on distributional recognition. Chehre provides a benchmark for evaluating diverse, dynamic, and distributional facial expression recognition

Problem

• facial expression recognition
• perceptual diversity
• dynamic expressions
• emoji-prompted
• distributional annotation

Innovation

• emoji-prompted
• dynamic facial expressions
• perceptual diversity
• anonymized synthetic faces
• distributional expression recognition