GraphRunner: A Multi-Stage Framework for Efficient and Accurate Graph-Based Retrieval

2025-07-11

AI Summary

Traditional RAG methods struggle to retrieve structured, interconnected data—such as knowledge graphs—due to their reliance on LLM-driven iterative single-hop traversal, which is prone to reasoning errors and hallucination. To address this, we propose GraphRunner, a graph retrieval framework based on a “plan–verify–execute” paradigm: (1) it generates high-level traversal actions for single-step multi-hop path planning; (2) it performs structural pre-verification of candidate paths using explicit graph constraints to detect and rectify LLM hallucinations; and (3) it safely executes the verified retrieval. This design jointly enhances reasoning fidelity and graph-structural awareness. On GRBench, GraphRunner achieves 10–50% higher retrieval accuracy than the strongest baseline, reduces inference cost by 3.0–12.9×, and cuts response latency by 2.5–7.1×—demonstrating substantial improvements in efficiency, robustness, and interpretability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Conventional Retrieval Augmented Generation (RAG) approaches are common in text-based applications. However, they struggle with structured, interconnected datasets like knowledge graphs, where understanding underlying relationships is crucial for accurate retrieval. A common direction in graph-based retrieval employs iterative, rule-based traversal guided by Large Language Models (LLMs). Such existing iterative methods typically combine reasoning with single hop traversal at each step, making them vulnerable to LLM reasoning errors and hallucinations that ultimately hinder the retrieval of relevant information. To address these limitations, we propose GraphRunner, a novel graph-based retrieval framework that operates in three distinct stages: planning, verification, and execution. This introduces high-level traversal actions that enable multi-hop exploration in a single step. It also generates a holistic traversal plan, which is verified against the graph structure and pre-defined traversal actions, reducing reasoning errors and detecting hallucinations before execution. GraphRunner significantly reduces LLM reasoning errors and detects hallucinations through validation. Our evaluation using the GRBench dataset shows that GraphRunner consistently outperforms existing approaches, achieving 10-50% performance improvements over the strongest baseline while reducing inference cost by 3.0-12.9x and response generation time by 2.5-7.1x, making it significantly more robust and efficient for graph-based retrieval tasks.

Problem

• Improves accuracy in graph-based retrieval by reducing LLM reasoning errors
• Enables multi-hop exploration in structured datasets like knowledge graphs
• Reduces inference costs and response time for graph-based tasks

Innovation

• Three-stage framework: planning, verification, execution
• Multi-hop exploration in single step
• Validates traversal plan to reduce errors