SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression

2026-04-20

AI Summary

This work addresses the challenges in conversational emotion recognition and consistent expression—namely, the scarcity of high-quality annotated data and the limitations of static modeling paradigms—by proposing SELF-EMO, a self-evolving framework. SELF-EMO introduces a novel joint modeling paradigm for emotion recognition and response generation, wherein the model simultaneously acts as both recognizer and responder through a role-based self-play mechanism. It iteratively produces diverse dialogue trajectories via auxiliary tasks and employs a data flywheel driven by a smoothed IoU-based reward signal alongside the SELF-GRPO reinforcement learning algorithm to enable unsupervised closed-loop optimization. Evaluated on IEMOCAP, MELD, and EmoryNLP benchmarks, the approach achieves state-of-the-art performance, improving accuracy by 6.33% and 8.54% with Qwen3-4B and Qwen3-8B backbones, respectively, thereby demonstrating its effectiveness and strong generalization capability.

Technology Category

• Humans and AI: Emotional Intelligence
• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Emotion Recognition in Conversation (ERC) has become a fundamental capability for large language models (LLMs) in human-centric interaction. Beyond accurate recognition, coherent emotional expression is also crucial, yet both are limited by the scarcity and static nature of high-quality annotated data. In this work, we propose SELF-EMO, a self-evolution framework grounded in the hypothesis that better emotion prediction leads to more consistent emotional responses. We introduce two auxiliary tasks, emotional understanding and emotional expression, and design a role-based self-play paradigm where the model acts as both an emotion recognizer and a dialogue responder. Through iterative interactions, the model generates diverse conversational trajectories, enabling scalable data generation. To ensure quality, we adopt a data flywheel mechanism that filters candidate predictions and responses using a smoothed IoU-based reward and feeds selected samples back for continuous self-improvement without external supervision. We further develop SELF-GRPO, a reinforcement learning algorithm that stabilizes optimization with multi-label alignment rewards and group-level consistency signals. Experiments on IEMOCAP, MELD, and EmoryNLP show that SELF-EMO achieves state-of-the-art performance, improving accuracy by +6.33% on Qwen3-4B and +8.54% on Qwen3-8B, demonstrating strong effectiveness and generalization.

Problem

• Emotion Recognition in Conversation
• Emotional Expression
• Data Scarcity
• Self-Evolution
• Consistent Response

Innovation

• self-evolution
• emotion recognition in conversation
• self-play
• data flywheel
• reinforcement learning