DecEx-RAG: Boosting Agentic Retrieval-Augmented Generation with Decision and Execution Optimization via Process Supervision

2025-10-07

AI Summary

To address low exploration efficiency, sparse rewards, and ambiguous global feedback in Agentic Retrieval-Augmented Generation (RAG) for complex tasks, this paper formalizes RAG as a two-stage decision-execution Markov Decision Process and proposes the DecEx-RAG framework. Its key contributions are: (1) process-level policy optimization with fine-grained procedural supervision, enabling autonomous task decomposition and dynamic retrieval; (2) an efficient pruning strategy to enhance both data expansion quality and construction speed; and (3) integration of reinforcement learning with a dynamic workflow mechanism. Evaluated on six benchmark datasets, DecEx-RAG achieves an average absolute performance gain of 6.2% and improves data construction efficiency by 5.8× over state-of-the-art baselines.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Agentic Retrieval-Augmented Generation (Agentic RAG) enhances the processing capability for complex tasks through dynamic retrieval and adaptive workflows. Recent advances (e.g., Search-R1) have shown that outcome-supervised reinforcement learning demonstrate strong performance. However, this approach still suffers from inefficient exploration, sparse reward signals, and ambiguous global reward feedback. To address these challenges, we propose DecEx-RAG, which models RAG as a Markov Decision Process (MDP) incorporating decision-making and execution, while introducing an efficient pruning strategy to optimize data expansion. Through comprehensive process-level policy optimization, DecEx-RAG significantly enhances the autonomous task decomposition, dynamic retrieval, and high-quality answer generation capabilities of large language models (LLMs). Experiments show that DecEx-RAG achieves an average absolute performance improvement of $6.2%$ across six datasets, significantly outperforming existing baselines. Moreover, the pruning strategy improves data construction efficiency by nearly $6 imes$, providing an efficient solution for process-supervised RAG training. The code is available at https://github.com/sdsxdxl/DecEx-RAG.

Problem

• Optimizes exploration and reward signals in agentic RAG
• Enhances autonomous task decomposition and dynamic retrieval
• Improves data construction efficiency via pruning strategy

Innovation

• Models RAG as Markov Decision Process with decision execution
• Introduces pruning strategy to optimize data expansion efficiency
• Uses process-level policy optimization for autonomous task decomposition