ARGen: Affect-Reinforced Generative Augmentation towards Vision-based Dynamic Emotion Perception

2026-04-14

AI Summary

This work addresses the challenge of in-the-wild dynamic facial expression recognition, which is hindered by data scarcity and long-tailed emotion distributions that impede modeling the temporal dynamics of rare emotional states. To overcome this limitation, the authors propose ARGen, a two-stage framework that enables data-adaptive generation of expressive facial videos through affective semantic injection followed by adaptive reinforcement-enhanced diffusion. The method integrates facial action units, vision-language models, and text-conditioned image-to-video diffusion models, while incorporating interpretable affective priors and a multi-objective reinforcement learning strategy. This approach substantially enhances both the naturalness of generated expressions and the accuracy of emotion recognition, achieving state-of-the-art performance in terms of generation quality and downstream recognition tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Dynamic facial expression recognition in the wild remains challenging due to data scarcity and long-tail distributions, which hinder models from effectively learning the temporal dynamics of scarce emotions. To address these limitations, we propose ARGen, an Affect-Reinforced Generative Augmentation Framework that enables data-adaptive dynamic expression generation for robust emotion perception. ARGen operates in two stages: Affective Semantic Injection (ASI) and Adaptive Reinforcement Diffusion (ARD). The ASI stage establishes affective knowledge alignment through facial Action Units and employs a retrieval-augmented prompt generation strategy to synthesize consistent and fine-grained affective descriptions via large-scale visual-language models, thereby injecting interpretable emotional priors into the generation process. The ARD stage integrates text-conditioned image-to-video diffusion with reinforcement learning, introducing inter-frame conditional guidance and a multi-objective reward function to jointly optimize expression naturalness, facial integrity, and generative efficiency. Extensive experiments on both generation and recognition tasks verify that ARGen substantially enhances synthesis fidelity and improves recognition performance, establishing an interpretable and generalizable generative augmentation paradigm for vision-based affective computing.

Problem

• dynamic facial expression recognition
• data scarcity
• long-tail distribution
• temporal dynamics
• emotion perception

Innovation

• Affect-Reinforced Generation
• Retrieval-Augmented Prompting
• Action Units Alignment
• Reinforcement Learning Diffusion
• Dynamic Facial Expression Synthesis