Multi-Step Semantic Reasoning in Generative Retrieval

2026-03-12

AI Summary

This work addresses the limited retrieval accuracy of existing generative retrieval models when handling complex queries—such as those in financial reports—that require multi-step numerical reasoning. To overcome this limitation, the authors propose ReasonGR, a novel framework that integrates task instructions with step-by-step reasoning guidance through structured prompt engineering. Additionally, ReasonGR incorporates a lightweight, reasoning-oriented parameter adaptation module to enhance the model’s semantic reasoning capabilities within numerical contexts. Experimental results on the FinQA dataset demonstrate that the proposed approach significantly improves both retrieval accuracy and result consistency, thereby validating its effectiveness and innovation in reasoning-intensive retrieval tasks.

Technology Category

• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Generative retrieval (GR) models encode a corpus within model parameters and generate relevant document identifiers directly for a given query. While this paradigm shows promise in retrieval tasks, existing GR models struggle with complex queries in numerical contexts, such as those involving semantic reasoning over financial reports, due to limited reasoning capabilities. This limitation leads to suboptimal retrieval accuracy and hinders practical applicability. We propose ReasonGR, a framework designed to enhance multi-step semantic reasoning in numerical contexts within GR. ReasonGR employs a structured prompting strategy combining task-specific instructions with stepwise reasoning guidance to better address complex retrieval queries. Additionally, it integrates a reasoning-focused adaptation module to improve the learning of reasoning-related parameters. Experiments on the FinQA dataset, which contains financial queries over complex documents, demonstrate that ReasonGR improves retrieval accuracy and consistency, indicating its potential for advancing GR models in reasoning-intensive retrieval scenarios.

Problem

• generative retrieval
• semantic reasoning
• numerical reasoning
• complex queries
• retrieval accuracy

Innovation

• Generative Retrieval
• Multi-Step Reasoning
• Structured Prompting
• Reasoning Adaptation
• Numerical Context