Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation

2024-10-28

AI Summary

Existing RAG methods (e.g., ReAct) embed reasoning chains directly into model context, leading to fragmented planning and execution failures that hinder multi-hop reasoning. To address this, we propose Plan*RAG—a novel framework that, for the first time, explicitly constructs an external directed acyclic graph (DAG) at test time as a structured reasoning plan. This enables systematic path exploration, generation of atomic subqueries, and parallelized retrieval execution, while context boundary control ensures reasoning integrity. Crucially, Plan*RAG decouples plan generation from execution, enabling plug-and-play integration with existing RAG systems. Evaluated on standard multi-hop benchmarks, Plan*RAG consistently outperforms state-of-the-art approaches—including RQ-RAG and Self-RAG—achieving significant gains in both reasoning accuracy and efficiency, with negligible increase in computational overhead.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Planning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce Plan*RAG, a novel framework that enables structured multi-hop reasoning in retrieval-augmented generation (RAG) through test-time reasoning plan generation. While existing approaches such as ReAct maintain reasoning chains within the language model's context window, we observe that this often leads to plan fragmentation and execution failures. Our key insight is that by isolating the reasoning plan as a directed acyclic graph (DAG) outside the LM's working memory, we can enable (1) systematic exploration of reasoning paths, (2) atomic subqueries enabling precise retrievals and grounding, and (3) efficiency through parallel execution and bounded context window utilization. Moreover, Plan*RAG's modular design allows it to be integrated with existing RAG methods, thus providing a practical solution to improve current RAG systems. On standard multi-hop reasoning benchmarks, Plan*RAG consistently achieves improvements over recently proposed methods such as RQ-RAG and Self-RAG, while maintaining comparable computational costs.

Problem

• Structured multi-hop reasoning in RAG
• Preventing plan fragmentation and execution failures
• Efficient parallel execution and context utilization

Innovation

• Structured multi-hop reasoning framework
• Isolated reasoning plan as DAG
• Modular design for RAG integration