Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs

2025-09-16

AI Summary

Existing graph reasoning approaches rely on task-specific fine-tuning or predefined algorithms, limiting generalizability and scalability. Method: We propose GRRAF, a zero-shot, training-free framework that integrates retrieval-augmented generation (RAG) with large language models’ (LLMs) code-generation capability. GRRAF prompts an LLM to directly generate executable graph query code (e.g., Cypher), executes it against a graph database to retrieve answers, and employs a timeout-constrained error-feedback loop for automatic correction and efficiency guarantees. Contribution/Results: GRRAF is the first framework enabling cross-task, scalable, zero-shot general graph reasoning—without task-specific training or handcrafted rules. It achieves 100% accuracy on most tasks in the GraphInstruct benchmark, supports graphs with up to ten thousand nodes, demonstrates superior subgraph matching performance, and incurs inference cost independent of graph size.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

We propose a new, training-free method, Graph Reasoning via Retrieval Augmented Framework (GRRAF), that harnesses retrieval-augmented generation (RAG) alongside the code-generation capabilities of large language models (LLMs) to address a wide range of graph reasoning tasks. In GRRAF, the target graph is stored in a graph database, and the LLM is prompted to generate executable code queries that retrieve the necessary information. This approach circumvents the limitations of existing methods that require extensive finetuning or depend on predefined algorithms, and it incorporates an error feedback loop with a time-out mechanism to ensure both correctness and efficiency. Experimental evaluations on the GraphInstruct dataset reveal that GRRAF achieves 100% accuracy on most graph reasoning tasks, including cycle detection, bipartite graph checks, shortest path computation, and maximum flow, while maintaining consistent token costs regardless of graph sizes. Imperfect but still very high performance is observed on subgraph matching. Notably, GRRAF scales effectively to large graphs with up to 10,000 nodes.

Problem

• Solving graph reasoning tasks without training using retrieval-augmented LLMs
• Overcoming limitations of fine-tuning requirements in graph algorithms
• Scaling graph reasoning to large graphs with thousands of nodes

Innovation

• Retrieval-augmented generation with LLMs
• Code-generation for graph queries
• Error feedback loop mechanism