R$^2$-Searcher: Calibrating Retrieval and Reasoning Boundaries for Agentic Search

2026-06-26

AI Summary

This work addresses the boundary misalignment between retrieval and reasoning in multi-hop question answering, which arises from incomplete or irrelevant retrieved evidence. To tackle this issue, the authors propose an explicit calibration mechanism that constructs precise reasoning contexts through fine-grained query-term-guided fact extraction, combined with a post-retrieval reflection module and a tree-based exploration strategy to dynamically refine the retrieval-reasoning boundary. Furthermore, they introduce R²PO, an end-to-end reinforcement learning algorithm that enables mutual enhancement and joint optimization of retrieval and reasoning processes. The proposed approach achieves state-of-the-art performance across seven challenging multi-hop QA benchmarks, significantly outperforming existing methods in both answer accuracy and the quality of retrieval-reasoning alignment.

Abstract

Recent search agents for multi-hop reasoning often fail by either retrieving incomplete evidence or reasoning over irrelevant portions of the retrieved content, leading to a retrieval-reasoning boundary shift. We propose R$^2$-Searcher, a novel framework that explicitly explores and calibrates the retrieval and reasoning boundaries via fine-grained, query-token-guided evidence modeling and post-retrieval reflection. Specifically, R$^2$-Searcher: (1) constructs fine-grained reasoning contexts by extracting precise facts from retrieved content based on query token semantics (e.g., subjects, actions, temporal markers, and degree modifiers), thereby guiding the attention of search agent; (2) introduces a retrieval reflection mechanism that evaluates and corrects boundary deviations after each retrieval step, guiding the generation of improved queries grounded in the extracted reasoning contexts; and (3) employs an end-to-end reasoning-reflection-guided reinforcement learning algorithm, R$^2$PO, which jointly optimizes both boundaries through a tree-based exploration of reasoning regions and reflections. Our method significantly enhances the quality of both retrieval and reasoning, establishing an iterative loop where retrieval and reasoning mutually enhance each other. Extensive experiments on seven complex multi-hop QA benchmarks demonstrate that R$^2$-Searcher significantly outperforms state-of-the-art agentic search methods in answer accuracy and retrieval-reasoning quality. Ablation studies further confirm the critical role of retrieval-reasoning boundary calibration.

Problem

• retrieval-reasoning boundary
• multi-hop reasoning
• agentic search
• evidence retrieval
• reasoning calibration

Innovation

• retrieval-reasoning boundary calibration
• query-token-guided evidence modeling
• retrieval reflection
• reasoning-reflection-guided reinforcement learning
• agentic search