Revealing the Challenge of Detecting Character Knowledge Errors in LLM Role-Playing

2024-09-18

arXiv.org

AI Summary

This work addresses the underexplored problem of distinguishing and detecting Known Knowledge Errors (KKEs) versus Unknown Knowledge Errors (UKEs) in role-playing by large language models (LLMs)—a critical challenge for role knowledge consistency. We introduce the first probe dataset specifically designed to assess role-knowledge consistency and conduct the first systematic evaluation of mainstream LLMs’ KKE/UKE detection capabilities, revealing pervasive deficiencies. To address this, we propose Agent-based Self-Recollection and Self-Doubt (S2RD), a novel reasoning paradigm integrating self-recollection and self-doubt mechanisms via multi-strategy chain-of-thought reasoning, significantly improving KKE/UKE detection accuracy. Experiments demonstrate that S2RD effectively alleviates performance bottlenecks; however, it also uncovers a fundamental limitation in LLMs’ ability to model intra-role knowledge boundaries. Our work establishes a key evaluation benchmark and technical pathway for automatically constructing high-quality, role-specific training corpora.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Representation Languages
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Large language model (LLM) role-playing has gained widespread attention, where the authentic character knowledge is crucial for constructing realistic LLM role-playing agents. However, existing works usually overlook the exploration of LLMs' ability to detect characters' known knowledge errors (KKE) and unknown knowledge errors (UKE) while playing roles, which would lead to low-quality automatic construction of character trainable corpus. In this paper, we propose a probing dataset to evaluate LLMs' ability to detect errors in KKE and UKE. The results indicate that even the latest LLMs struggle to effectively detect these two types of errors, especially when it comes to familiar knowledge. We experimented with various reasoning strategies and propose an agent-based reasoning method, Self-Recollection and Self-Doubt (S2RD), to further explore the potential for improving error detection capabilities. Experiments show that our method effectively improves the LLMs' ability to detect error character knowledge, but it remains an issue that requires ongoing attention.

Problem

• Detecting character knowledge errors in LLM role-playing
• Evaluating LLMs' ability to identify known and unknown knowledge errors
• Improving error detection in character knowledge for realistic role-playing

Innovation

• Proposes RoleKE-Bench for KKE and UKE evaluation
• Introduces agent-based reasoning method S2RD
• Improves LLMs' error detection capabilities