Graph-S3: Enhancing Agentic textual Graph Retrieval with Synthetic Stepwise Supervision

2025-10-01

AI Summary

This paper addresses the tension between graph retrieval in large text-graphs and the context-length limitations of large language models (LLMs). We propose an agent-based subgraph retrieval method. Our key contributions are: (1) a stepwise reward mechanism grounded in offline gold subgraphs, mitigating sparse feedback in reinforcement learning; and (2) a two-stage training framework coupled with an LLM-driven synthetic data pipeline, enabling precise multi-hop subgraph localization. Evaluated on three benchmark datasets, our method achieves average improvements of 8.1% in accuracy and 9.7% in F₁-score over seven strong baselines, with particularly pronounced gains on multi-hop tasks. The code will be publicly released.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

A significant portion of real-world data is inherently represented as textual graphs, and integrating these graphs into large language models (LLMs) is promising to enable complex graph-based question answering. However, a key challenge in LLM-based textual graph QA systems lies in graph retrieval, i.e., how to retrieve relevant content from large graphs that is sufficiently informative while remaining compact for the LLM context. Existing retrievers suffer from poor performance since they either rely on shallow embedding similarity or employ interactive retrieving policies that demand excessive data labeling and training cost. To address these issues, we present Graph-$S^3$, an agentic textual graph reasoning framework that employs an LLM-based retriever trained with synthetic stepwise supervision. Instead of rewarding the agent based on the final answers, which may lead to sparse and unstable training signals, we propose to closely evaluate each step of the retriever based on offline-extracted golden subgraphs. Our main techniques include a data synthesis pipeline to extract the golden subgraphs for reward generation and a two-stage training scheme to learn the interactive graph exploration policy based on the synthesized rewards. Based on extensive experiments on three common datasets in comparison with seven strong baselines, our approach achieves an average improvement of 8.1% in accuracy and 9.7% in F$_1$ score. The advantage is even higher in more complicated multi-hop reasoning tasks. Our code will be open-sourced.

Problem

• Improving graph retrieval for large language models
• Reducing data labeling costs in graph QA systems
• Enhancing multi-hop reasoning on textual graphs

Innovation

• Uses synthetic stepwise supervision for graph retrieval
• Trains agent with offline-extracted golden subgraphs
• Implements two-stage training for exploration policy