Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study over Open-ended Question Answering

2024-10-10

arXiv.org

AI Summary

This work addresses the severe hallucination and low factual reliability of large language models (LLMs) in open-domain question answering. Methodologically, we propose OKGQA—the first knowledge graph (KG)-enhanced LLM trustworthiness benchmark designed for realistic scenarios—along with its perturbed variant, OKGQA-P. We introduce a KG-LLM co-evaluation framework featuring a novel dual-benchmark paradigm, formalizing two quantitative metrics: hallucination rate and reasoning enhancement. Our approach integrates KG embedding-based retrieval, structured semantic alignment, and controllable noise generation. Empirical results demonstrate that KG augmentation significantly reduces LLM hallucinations (average reduction: 37%), though efficacy critically depends on KG quality and alignment precision. This work establishes a reproducible benchmark, provides failure-mode analysis grounded in empirical evidence, and offers principled methodological guidance for KG-augmented LLM research.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Question Answering

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent works integrating Knowledge Graphs (KGs) have led to promising improvements in enhancing the reasoning accuracy of Large Language Models (LLMs). However, current benchmarks focus mainly on closed-ended tasks, leaving a gap in the assessment of more complex real-world scenarios. This gap has also obscured the evaluation of KGs' potential to mitigate the problem of hallucination in LLMs. To fill the gap, we introduce OKGQA, a new benchmark specifically designed to assess LLMs enhanced with KGs under open-ended, real-world question answering scenarios. OKGQA is designed to closely reflect the complexities of practical applications using questions from different types, and incorporates specific metrics to measure both hallucination ratio and the enhancement in reasoning capabilities. To consider the scenario in which KGs may have varying levels of mistakes, we propose another benchmark variant OKGQA-P to assess model performance when the semantics and structure of KGs are deliberately perturbed and contaminated. OKGQA aims to (1) explore whether KGs can make LLMs more trustworthy in an open-ended setting, and (2) conduct a comparative analysis to shed light on method design. We believe that this study can facilitate a more complete performance comparison and encourage continuous improvement in integrating KGs with LLMs to reduce hallucination.

Problem

• Assessing KGs' impact on LLMs in open-ended QA
• Measuring hallucination reduction in LLMs with KGs
• Evaluating KGs' role in enhancing LLM trustworthiness

Innovation

• Integrates Knowledge Graphs with LLMs
• Introduces OKGQA benchmark
• Assesses hallucination and reasoning enhancement