ImpRAG: Retrieval-Augmented Generation with Implicit Queries

2025-06-02

AI Summary

Traditional RAG decouples retrieval from generation and relies on explicit queries, limiting generalization. This paper proposes a query-agnostic RAG paradigm built upon decoder-only language models. It employs a hierarchical architecture with shared parameters to jointly optimize retrieval and generation in a single forward pass. A two-stage implicit reasoning mechanism enables the model to internally formulate information needs without external queries. Crucially, perplexity serves as a unified, task-agnostic training objective, providing implicit supervision for retrieval. Evaluated on eight knowledge-intensive tasks, the method achieves average Exact Match gains of 3.6–11.5 points over prior task-specific baselines—without exposure to task-specific evaluation data—demonstrating significantly improved cross-format and cross-task generalization.

Technology Category

• Natural Language Processing: Question Answering
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-Augmented Generation (RAG) systems traditionally treat retrieval and generation as separate processes, requiring explicit textual queries to connect them. This separation can limit the ability of models to generalize across diverse tasks. In this work, we propose a query-free RAG system, named ImpRAG, which integrates retrieval and generation into a unified model. ImpRAG allows models to implicitly express their information needs, eliminating the need for human-specified queries. By dividing pretrained decoder-only language models into specialized layer groups, ImpRAG optimizes retrieval and generation tasks simultaneously. Our approach employs a two-stage inference process, using the same model parameters and forward pass for both retrieval and generation, thereby minimizing the disparity between retrievers and language models. Experiments on 8 knowledge-intensive tasks demonstrate that ImpRAG achieves 3.6-11.5 improvements in exact match scores on unseen tasks with diverse formats, highlighting its effectiveness in enabling models to articulate their own information needs and generalize across tasks. Our analysis underscores the importance of balancing retrieval and generation parameters and leveraging generation perplexities as retrieval training objectives for enhanced performance.

Problem

• Traditional RAG systems separate retrieval and generation processes
• Explicit queries limit model generalization across diverse tasks
• ImpRAG integrates retrieval and generation into a unified model

Innovation

• Query-free RAG system integrating retrieval and generation
• Specialized layer groups optimize retrieval and generation
• Two-stage inference with same model parameters