COMPEER: Controllable Empathetic Reinforcement Reasoning for Emotional Support Conversation

2025-08-13

AI Summary

Current empathic dialogue models for emotional support lack deep, psychology-grounded empathic reasoning capabilities, limiting their effectiveness in promoting users’ emotional well-being. To address this, we propose the Controllable Empathic Reasoning Framework (CERF), which integrates natural language inference with structured psychological steps—including emotion recognition, attribution analysis, and support strategy selection—to construct a fine-grained empathic process annotation dataset. We further design a process-and-outcome unified reward model, incorporating personality-aware dialogue rewriting and redundancy-aware reward reweighting to mitigate entropy collapse in reinforcement learning. Experiments demonstrate that CERF significantly improves response empathy, diversity, and human-likeness, outperforming state-of-the-art models across multiple emotional support benchmarks. Our work establishes a novel paradigm for interpretable and controllable psychological support dialogue systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: Conversational AI/Dialog Systems
• Humans and AI: Emotional Intelligence

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Emotional support conversations are crucial for promoting emotional well-being, yet current models often lack deep empathetic reasoning grounded in psychological principles. To address this, we propose controllable empathetic reasoning, which combines natural language reasoning with structured psychological steps. We construct a fine-grained dataset annotated with reasoning correctness and response preferences to enable this capability. To further enhance training, we employ reinforcement learning with a unified process-outcome reward model that delivers precise feedback. To mitigate response repetitiveness from entropy collapse, we introduce personality-based dialogue rewriting and a redundancy-aware reward reweighting strategy. Our approach significantly improves model's emotional support ability, advancing the development of empathetic, human-like support systems.

Problem

• Enhancing empathetic reasoning in emotional support conversations
• Combining natural language with psychological steps for better responses
• Reducing response repetitiveness using personality-based rewriting strategies

Innovation

• Combines natural language with psychological reasoning
• Uses reinforcement learning with unified rewards
• Introduces personality-based rewriting for diversity