EvolvTrip: Enhancing Literary Character Understanding with Temporal Theory-of-Mind Graphs

2025-06-16

AI Summary

Large language models (LLMs) struggle to model the dynamic evolution of characters’ mental states—such as beliefs, desires, and intentions—over extended narratives. Method: We propose EvolvTrip, the first fine-grained, temporally grounded mental knowledge graph explicitly designed for literary characters. We further introduce LitCharToM, a benchmark that systematically evaluates LLMs on four dimensions of Theory of Mind (ToM) reasoning—causality, temporality, counterfactuality, and multi-character interaction—under long-horizon narrative understanding. Our approach integrates graph-guided fine-tuning with a multi-dimensional evaluation framework. Results: Across multiple open- and closed-weight LLMs, our method significantly improves ToM reasoning performance, notably narrowing the capability gap between smaller and larger models. Moreover, it demonstrates strong robustness and compatibility in extended-context settings, establishing a new foundation for narrative-aware mental state modeling.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

A compelling portrayal of characters is essential to the success of narrative writing. For readers, appreciating a character's traits requires the ability to infer their evolving beliefs, desires, and intentions over the course of a complex storyline, a cognitive skill known as Theory-of-Mind (ToM). Performing ToM reasoning in prolonged narratives requires readers to integrate historical context with current narrative information, a task at which humans excel but Large Language Models (LLMs) often struggle. To systematically evaluate LLMs' ToM reasoning capability in long narratives, we construct LitCharToM, a benchmark of character-centric questions across four ToM dimensions from classic literature. Further, we introduce EvolvTrip, a perspective-aware temporal knowledge graph that tracks psychological development throughout narratives. Our experiments demonstrate that EvolvTrip consistently enhances performance of LLMs across varying scales, even in challenging extended-context scenarios. EvolvTrip proves to be particularly valuable for smaller models, partially bridging the performance gap with larger LLMs and showing great compatibility with lengthy narratives. Our findings highlight the importance of explicit representation of temporal character mental states in narrative comprehension and offer a foundation for more sophisticated character understanding. Our data and code are publicly available at https://github.com/Bernard-Yang/EvolvTrip.

Problem

• Evaluating LLMs' Theory-of-Mind reasoning in long narratives
• Tracking character psychological development in complex storylines
• Enhancing narrative comprehension with temporal mental state representation

Innovation

• Temporal Theory-of-Mind Graphs track character psychology
• LitCharToM benchmark evaluates LLMs' ToM reasoning
• EvolvTrip enhances LLM performance in narratives