RoleRAG: Enhancing LLM Role-Playing via Graph Guided Retrieval

2025-05-24

AI Summary

Large language models (LLMs) frequently exhibit knowledge inconsistency and hallucination in role-playing due to entity ambiguity and ill-defined cognitive boundaries. To address this, we propose a graph-guided retrieval-augmented framework. Our method integrates structured knowledge graph retrieval, boundary-aware re-ranking, and a retrieval-augmented generation (RAG) architecture. Key innovations include: (1) an entity-disambiguation-driven role knowledge indexing mechanism that precisely retrieves role-specific factual knowledge; and (2) a boundary-aware retriever that explicitly models each role’s epistemic scope and constrains generation accordingly. Evaluated on a multi-role benchmark, our approach significantly improves knowledge consistency (+18.7%) and reduces hallucination rate (−32.4%). It is compatible with both general-purpose and role-specialized LLMs. This work establishes a novel paradigm for controllable, faithful role simulation grounded in structured, boundary-respecting knowledge retrieval.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) have shown promise in character imitation, enabling immersive and engaging conversations. However, they often generate content that is irrelevant or inconsistent with a character's background. We attribute these failures to: (1) the inability to accurately recall character-specific knowledge due to entity ambiguity, and (2) a lack of awareness of the character's cognitive boundaries. To address these issues, we propose RoleRAG, a retrieval-based framework that integrates efficient entity disambiguation for knowledge indexing with a boundary-aware retriever for extracting contextually appropriate information from a structured knowledge graph. Experiments on role-playing benchmarks show that RoleRAG's calibrated retrieval helps both general-purpose and role-specific LLMs better align with character knowledge and reduce hallucinated responses.

Problem

• LLMs struggle with character-specific knowledge recall due to entity ambiguity
• LLMs lack awareness of character cognitive boundaries in role-playing
• Need for contextually appropriate retrieval from structured knowledge graphs

Innovation

• Graph guided retrieval for role-playing
• Entity disambiguation for knowledge indexing
• Boundary-aware retriever for contextual information