Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference

2026-06-18

AI Summary

This work addresses the challenge of knowledge conflicts that arise when large language models integrate internal knowledge with external context. Existing approaches typically resolve such conflicts by relying solely on one source, thereby failing to actively reconcile inconsistencies. To overcome this limitation, the paper proposes MACR, a novel framework that moves beyond the conventional binary choice paradigm by introducing an explicit conflict resolution mechanism. MACR employs a semantic entropy–enhanced confidence estimation to adaptively quantify the reliability of knowledge sources and integrates a multi-agent collaborative reasoning architecture encompassing rule induction, conflict analysis, and consistency coordination. This enables interpretable identification and resolution of conflicts. Experimental results demonstrate that MACR significantly outperforms state-of-the-art methods across multiple benchmarks while producing transparent and traceable solutions.

Abstract

Large language models (LLMs) have achieved strong performance across a wide range of language-based tasks by leveraging both extensive parametric knowledge and in-context learning ability, enabling them to incorporate external information provided in the input prompt. However, the integration of external knowledge can introduce conflicts, not only between the model's internal parametric knowledge and the external information, but also among multiple pieces of external contexts. Existing approaches typically assume that either the model or the provided context is reliable, overlooking the possibility that both sources may contain errors, and avoid conflicts by privileging one source over the other, rather than actively resolving inconsistencies. To address these limitations, we propose a novel framework MACR for LLM knowledge conflict resolution that moves beyond the conventional binary choice paradigm and incorporates an explicit conflict-resolution mechanism based on a multi-agent reasoning approach. Specifically, we first propose an adaptive knowledge assessment and retrieval approach that employs a modified semantic entropy measure to quantify an LLM's confidence in its answer to a given query. Based on this confidence estimation, MACR either externalizes the model's internal knowledge as textual representations or retrieves relevant external knowledge when internal knowledge is insufficient, generating basic contexts for subsequent reasoning. Then we introduce an inductive multi-agent reasoning framework with three specialized agents that, respectively, induce explicit rules, analyze potential conflicts, and resolve inconsistencies across all available contexts. Empirical results demonstrate that MACR significantly outperforms state-of-the-art baselines across benchmarks, while also providing interpretable resolutions of explicit conflicts.

Problem

• knowledge conflict
• large language models
• parametric knowledge
• contextual knowledge
• conflict resolution

Innovation

• knowledge conflict resolution
• multi-agent reasoning
• semantic entropy
• adaptive knowledge retrieval
• interpretable LLM inference