AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Construction

2025-10-17

AI Summary

Knowledge graph (KG) construction has long been decoupled from downstream retrieval-augmented generation (RAG) tasks, resulting in graph structures poorly aligned with practical question-answering requirements. Method: We propose the first RAG-effectiveness-driven, end-to-end KG construction framework. It formulates KG generation as a reinforcement learning policy optimization problem, employing a large language model (LLM) as the graph constructor and introducing two task-aware reward functions—answer accuracy and retrieval relevance—to guide structural learning. Contribution/Results: Our approach shifts the paradigm from “building a good graph” to “building a useful graph” for RAG. Evaluated on multiple open-domain QA benchmarks, it significantly improves KG-enhanced RAG performance over strong baselines. Empirical results validate both the effectiveness and state-of-the-art nature of task-driven KG structure learning.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Graph-based Machine Learning

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Building effective knowledge graphs (KGs) for Retrieval-Augmented Generation (RAG) is pivotal for advancing question answering (QA) systems. However, its effectiveness is hindered by a fundamental disconnect: the knowledge graph (KG) construction process is decoupled from its downstream application, yielding suboptimal graph structures. To bridge this gap, we introduce AutoGraph-R1, the first framework to directly optimize KG construction for task performance using Reinforcement Learning (RL). AutoGraph-R1 trains an LLM constructor by framing graph generation as a policy learning problem, where the reward is derived from the graph's functional utility in a RAG pipeline. We design two novel, task-aware reward functions, one for graphs as knowledge carriers and another as knowledge indices. Across multiple QA benchmarks, AutoGraph-R1 consistently enables graph RAG methods to achieve significant performance gains over using task-agnostic baseline graphs. Our work shows it is possible to close the loop between construction and application, shifting the paradigm from building intrinsically ``good'' graphs to building demonstrably ``useful'' ones.

Problem

• Bridging the disconnect between KG construction and downstream QA applications
• Optimizing knowledge graph structures for task performance using Reinforcement Learning
• Shifting from building intrinsically good graphs to demonstrably useful ones

Innovation

• Reinforcement Learning optimizes Knowledge Graph construction
• Task-aware reward functions guide graph generation process
• End-to-end training bridges construction and application gap