Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation

2024-12-24

AI Summary

To address critical challenges in knowledge graph question answering—including LLM hallucination, outdated knowledge, and noise from multi-source retrieval—this paper proposes the Adaptive Multi-Aspect Retrieval (AMAR) framework. AMAR integrates prompt-embedding-driven multi-view retrieval with retrieval-augmented generation (RAG). Its key contributions are: (1) a novel self-alignment module that enables cross-modal semantic alignment between textual prompts and structured knowledge (e.g., entities, relations, subgraphs); and (2) a relevance-aware soft gating mechanism that dynamically weights and fuses multi-granularity retrieved knowledge, suppressing noise while selectively injecting salient facts. Evaluated on WebQSP and ComplexWebQuestions (CWQ), AMAR achieves state-of-the-art performance: +1.9% absolute improvement in answer accuracy and +6.6% in logical form generation accuracy, significantly enhancing factual consistency and robustness in multi-hop reasoning.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) demonstrate remarkable capabilities, yet struggle with hallucination and outdated knowledge when tasked with complex knowledge reasoning, resulting in factually incorrect outputs. Previous studies have attempted to mitigate it by retrieving factual knowledge from large-scale knowledge graphs (KGs) to assist LLMs in logical reasoning and prediction of answers. However, this kind of approach often introduces noise and irrelevant data, especially in situations with extensive context from multiple knowledge aspects. In this way, LLM attention can be potentially mislead from question and relevant information. In our study, we introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. The Amar framework comprises two key sub-components: 1) a self-alignment module that aligns commonalities among entities, relations, and subgraphs to enhance retrieved text, thereby reducing noise interference; 2) a relevance gating module that employs a soft gate to learn the relevance score between question and multi-aspect retrieved data, to determine which information should be used to enhance LLMs' output, or even filtered altogether. Our method has achieved state-of-the-art performance on two common datasets, WebQSP and CWQ, showing a 1.9% improvement in accuracy over its best competitor and a 6.6% improvement in logical form generation over a method that directly uses retrieved text as context prompts. These results demonstrate the effectiveness of Amar in improving the reasoning of LLMs.

Problem

• Large Language Models
• Factuality Issues
• Interdisciplinary Knowledge

Innovation

• Amar Framework
• Knowledge Graph Processing
• Enhanced Logical Reasoning