BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering

2025-07-05

AI Summary

Knowledge Graph Question Answering (KGQA) faces persistent challenges—including inaccurate entity linking, sensitivity to traversal initialization, and poor generalization—when applied to custom knowledge graphs exhibiting structural heterogeneity and semantic variability. This paper proposes LLM-GT, an iterative KGQA framework that synergistically integrates large language models (LLMs) with domain-specific graph retrieval tools. Its core innovation lies in leveraging LLMs to generate structured intermediate artifacts—such as candidate entities, reasoning paths, and OpenCypher queries—which are then rigorously validated and refined via precise execution on graph databases, establishing a “generate–verify–optimize” closed loop. This design markedly reduces dependency on initial node selection and enhances cross-graph generalization. Evaluated on five heterogeneous benchmarks, LLM-GT achieves an average 4.5-percentage-point improvement over the second-best method and demonstrates strong robustness on custom graphs. The implementation is publicly available.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Knowledge graph question answering (KGQA) presents significant challenges due to the structural and semantic variations across input graphs. Existing works rely on Large Language Model (LLM) agents for graph traversal and retrieval; an approach that is sensitive to traversal initialization, as it is prone to entity linking errors and may not generalize well to custom ("bring-your-own") KGs. We introduce BYOKG-RAG, a framework that enhances KGQA by synergistically combining LLMs with specialized graph retrieval tools. In BYOKG-RAG, LLMs generate critical graph artifacts (question entities, candidate answers, reasoning paths, and OpenCypher queries), and graph tools link these artifacts to the KG and retrieve relevant graph context. The retrieved context enables the LLM to iteratively refine its graph linking and retrieval, before final answer generation. By retrieving context from different graph tools, BYOKG-RAG offers a more general and robust solution for QA over custom KGs. Through experiments on five benchmarks spanning diverse KG types, we demonstrate that BYOKG-RAG outperforms the second-best graph retrieval method by 4.5% points while showing better generalization to custom KGs. BYOKG-RAG framework is open-sourced at https://github.com/awslabs/graphrag-toolkit.

Problem

• Addresses structural and semantic variations in KGQA
• Reduces entity linking errors in custom knowledge graphs
• Improves generalization and robustness in graph retrieval

Innovation

• Combines LLMs with graph retrieval tools
• Generates graph artifacts for KG linking
• Retrieves context to refine answer generation