Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs

2025-10-09

AI Summary

Large language models (LLMs) suffer from factual omissions, hallucinations, and knowledge staleness in knowledge-intensive multi-hop question answering; meanwhile, existing knowledge graph question answering (KGQA) approaches are hindered by SPARQL query fragility, noisy subgraph retrieval, or exponential search-space explosion. This paper proposes an iterative, knowledge graph–based reasoning framework centered on an “Observe–Navigate” mechanism: a single, lightweight Search function dynamically explores reasoning paths without requiring predefined queries or large-scale subgraph retrieval, enabling robust adaptation to heterogeneous graph structures. Additionally, we introduce an adaptive filtering strategy that supports multi-hop relation probing and higher-order node reasoning. Evaluated across six benchmarks, our method achieves state-of-the-art performance—improving accuracy by 16% on Wikidata tasks and significantly outperforming prior approaches on Freebase.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Question Answering

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large language models (LLMs) have demonstrated impressive reasoning abilities yet remain unreliable on knowledge-intensive, multi-hop questions -- they miss long-tail facts, hallucinate when uncertain, and their internal knowledge lags behind real-world change. Knowledge graphs (KGs) offer a structured source of relational evidence, but existing KGQA methods face fundamental trade-offs: compiling complete SPARQL queries without knowing available relations proves brittle, retrieving large subgraphs introduces noise, and complex agent frameworks with parallel exploration exponentially expand search spaces. To address these limitations, we propose Search-on-Graph (SoG), a simple yet effective framework that enables LLMs to perform iterative informed graph navigation using a single, carefully designed extsc{Search} function. Rather than pre-planning paths or retrieving large subgraphs, SoG follows an ``observe-then-navigate'' principle: at each step, the LLM examines actual available relations from the current entity before deciding on the next hop. This approach further adapts seamlessly to different KG schemas and handles high-degree nodes through adaptive filtering. Across six KGQA benchmarks spanning Freebase and Wikidata, SoG achieves state-of-the-art performance without fine-tuning. We demonstrate particularly strong gains on Wikidata benchmarks (+16% improvement over previous best methods) alongside consistent improvements on Freebase benchmarks.

Problem

• Addressing LLM unreliability on knowledge-intensive multi-hop questions
• Overcoming brittleness and noise in existing KGQA methods
• Enabling iterative informed graph navigation using single Search function

Innovation

• Iterative informed graph navigation using single Search function
• Observe-then-navigate principle examining available relations
• Adaptive filtering for different KG schemas and high-degree nodes