CAREBench: Evaluating LLMs' Emotion Understanding by Assessing Cognitive Appraisal Reasoning

2026-05-16

AI Summary

Current evaluations of emotional understanding in large language models predominantly rely on discrete emotion labels, overlooking the underlying cognitive mechanisms of emotion generation. Addressing this gap, this work introduces CAREBench, a novel benchmark grounded in appraisal theory, which features multi-perspective (first- and third-person) narrative data annotated with complete cognitive appraisal reasoning chains. The dataset systematically captures the inference process from cognitive appraisals and valence ratings to multi-label emotions, enabling a process-level evaluation framework. Experimental results reveal that while strong models approach or even surpass human performance on certain tasks, they still exhibit notable deficiencies in appraisal-based reasoning and recognition of positive emotions. Moreover, conventional downstream metrics may overestimate models’ genuine capacity for emotional understanding.

Abstract

Emotion understanding is a core capability for LLMs to interact effectively with humans, yet existing evaluation paradigms rely on discrete emotion label prediction and fail to capture the cognitive processes underlying emotion generation. Grounded in appraisal theory, we introduce CAREBench, the first benchmark with complete inferential chain annotations from both first- and third-person perspectives on real-world narratives, spanning appraisal reasoning, appraisal ratings, and multi-label emotion annotation. We propose a process-level evaluation framework and conduct systematic experiments across six LLMs organized around four research questions. We find that stronger models match or surpass human observers on certain tasks, yet fall short on appraisal reasoning and positive emotion recognition; performance across chain steps and sensitivity to appraisal interventions exhibit dissociations across models; and current models have not internalized the mechanisms needed to capture human subjective heterogeneity. These findings suggest that downstream emotion prediction metrics may overestimate LLMs' true emotion understanding, and CAREBench provides a foundation for more diagnostically informative evaluation of LLMs' affective cognitive capabilities.

Problem

• emotion understanding
• cognitive appraisal
• LLM evaluation
• appraisal theory
• affective cognition

Innovation

• CAREBench
• cognitive appraisal reasoning
• emotion understanding
• process-level evaluation
• LLM affective cognition