MPT: Motion Prompt Tuning for Micro-Expression Recognition

2025-08-12

AI Summary

Micro-expression recognition (MER) faces two key challenges: severe scarcity of labeled data and the inability of existing large models to effectively capture transient, subtle facial movements. To address these, this work introduces prompt learning to MER for the first time, proposing Motion Prompt Tuning (MPT). MPT enhances dynamic features via motion amplification, employs Gaussian tokenization to generate temporally aware motion prompts, and incorporates a grouped adapter to improve the large model’s adaptability to discriminative micro-expression representations. Crucially, MPT requires no modification to the backbone architecture and achieves efficient few-shot transfer learning using only lightweight, plug-and-play modules. Extensive experiments on three benchmark datasets—CASME II, SAMM, and SMIC—demonstrate significant improvements over state-of-the-art methods, validating both effectiveness and cross-dataset generalizability.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Micro-expression recognition (MER) is crucial in the affective computing field due to its wide application in medical diagnosis, lie detection, and criminal investigation. Despite its significance, obtaining micro-expression (ME) annotations is challenging due to the expertise required from psychological professionals. Consequently, ME datasets often suffer from a scarcity of training samples, severely constraining the learning of MER models. While current large pre-training models (LMs) offer general and discriminative representations, their direct application to MER is hindered by an inability to capture transitory and subtle facial movements-essential elements for effective MER. This paper introduces Motion Prompt Tuning (MPT) as a novel approach to adapting LMs for MER, representing a pioneering method for subtle motion prompt tuning. Particularly, we introduce motion prompt generation, including motion magnification and Gaussian tokenization, to extract subtle motions as prompts for LMs. Additionally, a group adapter is carefully designed and inserted into the LM to enhance it in the target MER domain, facilitating a more nuanced distinction of ME representation. Furthermore, extensive experiments conducted on three widely used MER datasets demonstrate that our proposed MPT consistently surpasses state-of-the-art approaches and verifies its effectiveness.

Problem

• Addresses micro-expression recognition with limited training samples
• Adapts large pre-training models for subtle facial motion analysis
• Overcomes challenges in capturing transient micro-expression features

Innovation

• Motion Prompt Tuning adapts LMs
• Motion magnification extracts subtle motions
• Group adapter enhances MER representation