What Is Seen Cannot Be Unseen: The Disruptive Effect of Knowledge Conflict on Large Language Models

2025-06-06

AI Summary

This work investigates the behavioral mechanisms of large language models (LLMs) when contextual information conflicts with parametric knowledge. To address this knowledge conflict problem, we propose the first scalable diagnostic framework, comprising a multi-task benchmark dataset, controllable prompt engineering, conflict-aware data construction, attribution-based reasoning interventions, and multi-task performance analysis. Our findings reveal: (1) conflicts significantly impair only external-knowledge-dependent tasks; (2) optimal performance occurs when context and parametric knowledge align; (3) models struggle to fully suppress internal beliefs, yet providing explicit conflict explanations substantially increases contextual adoption; and (4) rationale generation unexpectedly strengthens contextual reliance—uncovering several counterintuitive phenomena. Collectively, these results establish a novel paradigm for understanding how LLMs integrate heterogeneous knowledge sources and perform trustworthy, context-sensitive reasoning.

Technology Category

• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models frequently rely on both contextual input and parametric knowledge to perform tasks. However, these sources can come into conflict, especially when retrieved documents contradict the model's parametric knowledge. We propose a diagnostic framework to systematically evaluate LLM behavior under context-memory conflict, where the contextual information diverges from their parametric beliefs. We construct diagnostic data that elicit these conflicts and analyze model performance across multiple task types. Our findings reveal that (1) knowledge conflict has minimal impact on tasks that do not require knowledge utilization, (2) model performance is consistently higher when contextual and parametric knowledge are aligned, (3) models are unable to fully suppress their internal knowledge even when instructed, and (4) providing rationales that explain the conflict increases reliance on contexts. These insights raise concerns about the validity of model-based evaluation and underscore the need to account for knowledge conflict in the deployment of LLMs.

Problem

• Evaluates LLM behavior under context-memory knowledge conflict
• Analyzes model performance when context contradicts parametric beliefs
• Assesses impact of knowledge conflict on task validity

Innovation

• Diagnostic framework for context-memory conflict
• Constructed data eliciting knowledge conflicts
• Rationales increase context reliance