Reliability Across Parametric and External Knowledge: Understanding Knowledge Handling in LLMs

2025-02-19

AI Summary

This paper addresses critical reliability challenges in large language models (LLMs) when integrating parametric knowledge with external knowledge—specifically, knowledge conflict resolution, robustness to noisy external information, and principled abstention under knowledge gaps. To this end, we propose the first dual-dimensional evaluation framework, orthogonalizing *parametric knowledge existence* against *external knowledge informativeness*. We construct a knowledge-sensitive benchmark, design controlled comparative experiments, and introduce knowledge-reliability–aware supervised fine-tuning. Our analysis systematically uncovers cross-scenario knowledge processing biases and their inter-scenario capability transfer patterns. Empirical results demonstrate that data augmentation guided by this framework significantly improves decision accuracy under knowledge conflicts and noise, and enhances the reasonableness of abstention under knowledge insufficiency—validating the effectiveness of structured evaluation for robust knowledge integration.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) enhance their problem-solving capability by leveraging both parametric and external knowledge. Beyond leveraging external knowledge to improve response accuracy, they require key capabilities for reliable knowledge-handling: resolving conflicts between knowledge sources, avoiding distraction from uninformative external knowledge, and abstaining when sufficient knowledge is unavailable. Prior studies have examined these scenarios in isolation or with limited scope. To systematically evaluate these capabilities, we introduce a comprehensive framework for analyzing knowledge-handling based on two key dimensions: the presence of parametric knowledge and the informativeness of external knowledge. Through analysis, we identify biases in knowledge utilization and examine how the ability to handle one scenario impacts performance in others. Furthermore, we demonstrate that training on data constructed based on the knowledge-handling scenarios improves LLMs' reliability in integrating and utilizing knowledge.

Problem

• Reliability in knowledge integration
• Handling conflicts between knowledge sources
• Improving LLMs' response accuracy

Innovation

• Leveraging parametric and external knowledge
• Systematic framework for knowledge-handling
• Training improves reliability in knowledge integration