Reinforced Information Retrieval

2025-02-17

AI Summary

To address the challenge of strong domain-knowledge dependency in professional-scenario cross-domain information retrieval and the limited adaptability of existing methods, this paper proposes Reinforced-IR: an unsupervised domain adaptation framework that jointly optimizes a pretrained retriever and a generator in an end-to-end manner. Its core innovation is the Self-Boosting Collaborative Reinforcement mechanism—where the retriever and generator iteratively refine each other via self-feedback, enabling precise target-domain adaptation without human annotations. The method integrates a DPR-style retriever, a large language model (LLM) generator, and reinforcement learning, augmented with self-supervised contrastive learning and gradient-coordinated parameter updates. Evaluated on multiple domain-specific benchmarks, Reinforced-IR achieves an average 12.6% improvement in retrieval accuracy over state-of-the-art domain adaptation approaches, demonstrating superior generalizability and low deployment overhead.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

While retrieval techniques are widely used in practice, they still face significant challenges in cross-domain scenarios. Recently, generation-augmented methods have emerged as a promising solution to this problem. These methods enhance raw queries by incorporating additional information from an LLM-based generator, facilitating more direct retrieval of relevant documents. However, existing methods struggle with highly specialized situations that require extensive domain expertise. To address this problem, we present extbf{Reinforced-IR}, a novel approach that jointly adapts a pre-trained retriever and generator for precise cross-domain retrieval. A key innovation of Reinforced-IR is its extbf{Self-Boosting} framework, which enables retriever and generator to learn from each other's feedback. Specifically, the generator is reinforced to generate query augmentations that enhance the retriever's performance, while the retriever is trained to better discriminate the relevant documents identified by the generator. This iterative process allows the end-to-end retrieval performance to be progressively optimized using an unlabeled corpus from the target domain. In our experiment, Reinforced-IR outperforms existing domain adaptation methods by a large margin, leading to substantial improvements in retrieval quality across a wide range of application scenarios.

Problem

• Improves cross-domain retrieval accuracy
• Adapts retriever and generator jointly
• Optimizes retrieval with unlabeled domain data

Innovation

• LLM-based generator enhancement
• Self-Boosting feedback framework
• Cross-domain retrieval optimization