GPR: Empowering Generation with Graph-Pretrained Retriever

2025-05-30

AI Summary

Existing knowledge graph–based retrieval-augmented generation (GRAG) methods suffer from domain misalignment and insufficient graph-structural awareness. To address these challenges, we propose GPR—the first end-to-end pre-trained graph retriever operating directly over knowledge graphs. Our approach introduces three key innovations: (1) LLM-guided graph data augmentation to synthesize high-quality subgraph queries; (2) a structure-aware contrastive learning objective enabling fine-grained subgraph matching; and (3) joint multi-hop subgraph encoding with cross-modal alignment. Evaluated across two GRAG benchmarks, three mainstream large language models, and five baseline retrievers, GPR consistently achieves significant improvements in retrieval accuracy and downstream generation quality. Results demonstrate that GPR serves as a robust, structure-aware GRAG retrieval component with strong generalization capability across domains and model architectures.

Technology Category

• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Graph retrieval-augmented generation (GRAG) places high demands on graph-specific retrievers. However, existing retrievers often rely on language models pretrained on plain text, limiting their effectiveness due to domain misalignment and structure ignorance. To address these challenges, we propose GPR, a graph-based retriever pretrained directly on knowledge graphs. GPR aligns natural language questions with relevant subgraphs through LLM-guided graph augmentation and employs a structure-aware objective to learn fine-grained retrieval strategies. Experiments on two datasets, three LLM backbones, and five baselines show that GPR consistently improves both retrieval quality and downstream generation, demonstrating its effectiveness as a robust retrieval solution for GRAG.

Problem

• Improving graph retrieval for generation tasks
• Addressing domain misalignment in graph retrievers
• Enhancing structure awareness in retrieval strategies

Innovation

• Pretrains retriever directly on knowledge graphs
• Uses LLM-guided graph augmentation for alignment
• Employs structure-aware objective for fine-grained retrieval