Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning

2025-09-30

AI Summary

Current KG-RAG systems rely on multi-LLM modular pipelines (e.g., planning, reasoning, generation), incurring high inference costs and tight coupling to specific knowledge graphs (KGs). To address this, we propose KG-R1: a reinforcement learning–based, single-agent KG-RAG framework that performs end-to-end interactive retrieval and reasoning for dynamic path exploration and response generation. Its core innovation lies in replacing rigid multi-module pipelines with a unified agent architecture, optimized via RL to enable cross-KG transferability and plug-and-play deployment—built upon Qwen-2.5-3B. Experiments on multiple KG question-answering benchmarks demonstrate that KG-R1 achieves superior accuracy using significantly fewer generated tokens than larger multi-module baselines. Moreover, it exhibits strong generalization to unseen KGs, confirming its robustness and adaptability across diverse knowledge sources.

Technology Category

• Knowledge Representation and Reasoning: Knowledge Representation Languages
• Multiagent Systems: Multiagent Learning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Knowledge-graph retrieval-augmented generation (KG-RAG) couples large language models (LLMs) with structured, verifiable knowledge graphs (KGs) to reduce hallucinations and expose reasoning traces. However, many KG-RAG systems compose multiple LLM modules (e.g planning, reasoning, and responding), inflating inference cost and binding behavior to a specific target KG. To address this, we introduce KG-R1, an agentic KG retrieval-augmented generation (KG-RAG) framework through reinforcement learning (RL). KG-R1 utilizes a single agent that interacts with KGs as its environment, learning to retrieve at each step and incorporating the retrieved information into its reasoning and generation. The process is optimized through end-to-end RL. In controlled experiments across Knowledge-Graph Question Answering (KGQA) benchmarks, our method demonstrates both efficiency and transferability: Using Qwen-2.5-3B, KG-R1 improves answer accuracy with fewer generation tokens than prior multi-module workflow methods that use larger foundation or fine-tuned models. Furthermore, KG-R1 enables plug and play: after training, it maintains strong accuracy on new KGs without modification. These properties make KG-R1 a promising KG-RAG framework for real-world deployment. Our code is publicly available at https://github.com/Jinyeop3110/KG-R1.

Problem

• Reduces hallucinations in LLMs using knowledge graphs
• Optimizes KG-RAG efficiency through reinforcement learning
• Enables transferability across knowledge graphs without modification

Innovation

• Single agent framework via reinforcement learning
• End-to-end optimization for KG retrieval and generation
• Transferable across knowledge graphs without modification