AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition

2026-05-07

AI Summary

This study addresses the challenge in open-vocabulary multimodal emotion recognition (OV-MER) where conventional token-wise losses are misaligned with the non-differentiable emotion wheel metrics. To bridge this gap, the authors introduce AffectGPT-RL, the first framework to incorporate reinforcement learning into OV-MER, featuring a reward mechanism explicitly aligned with emotion wheel evaluation criteria to optimize the outputs of generative large language models. By integrating multimodal information with a tailored reinforcement learning strategy, the proposed method achieves significant performance gains on OV-MER and establishes a new state-of-the-art on the basic emotion recognition task within the MER-UniBench benchmark. The work further provides systematic insights into the pivotal roles of reinforcement learning in inference optimization, reward design, and cross-task generalization.

Abstract

Open-Vocabulary Multimodal Emotion Recognition (OV-MER) aims to predict emotions without being constrained by predefined label spaces, thereby enabling fine-grained emotion understanding. Unlike traditional discriminative methods, OV-MER leverages generative models to capture the full spectrum of emotions and employs emotion wheels (EWs) for metric calculation. Previous approaches primarily rely on token-level loss during training. However, this objective is misaligned with the metrics used in OV-MER, and these metrics cannot be directly optimized via gradient backpropagation. To address this limitation, we turn our attention to reinforcement learning, as this strategy can optimize non-differentiable objectives. We term this framework AffectGPT-RL. Furthermore, we conduct extensive experiments to elucidate the role of reinforcement learning in this task, revealing the necessity of the reasoning process, the impact of different rewards, and the generalizability to other emotion tasks such as sentiment analysis and basic emotion recognition. Experimental results demonstrate that AffectGPT-RL yields significant performance improvements on OV-MER. Beyond this task, we also achieve remarkable performance gains on basic emotion recognition, attaining state-of-the-art results on MER-UniBench. To the best of our knowledge, this is the pioneering work exploring the role of reinforcement learning in OV-MER, providing valuable guidance for subsequent researchers. Our code is provided in the supplementary material and will be released to facilitate future research.

Problem

• Open-Vocabulary Emotion Recognition
• Reinforcement Learning
• Non-differentiable Metrics
• Multimodal Emotion Recognition
• Emotion Wheels

Innovation

• Reinforcement Learning
• Open-Vocabulary Emotion Recognition
• Multimodal Emotion Recognition
• Non-differentiable Optimization
• Emotion Wheel