How Generative IR Retrieves Documents Mechanistically

2025-03-25

AI Summary

This work addresses the opacity of internal mechanisms in generative information retrieval (GenIR) models. We introduce mechanistic interpretability methods—specifically patching interventions, vocabulary projection, and constrained encoder replacement—to systematically analyze end-to-end document ranking in Transformer encoder-decoder architectures. Our empirical analysis reveals that the decoder dominates retrieval, with the process decomposable into three sequential phases: warming-up, bridging, and interaction. Query-document interaction occurs exclusively in the final phase; the bridging phase relies primarily on cross-attention, while the interaction phase depends critically on MLP layers. These findings uncover the computational division of labor and information flow pathways within GenIR models, establishing the decoder’s central role in retrieval. Beyond theoretical insight, our results provide actionable foundations for model diagnosis, controllable editing, and parameter-efficient design.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Generative Information Retrieval (GenIR) is a novel paradigm in which a transformer encoder-decoder model predicts document rankings based on a query in an end-to-end fashion. These GenIR models have received significant attention due to their simple retrieval architecture while maintaining high retrieval effectiveness. However, in contrast to established retrieval architectures like cross-encoders or bi-encoders, their internal computations remain largely unknown. Therefore, this work studies the internal retrieval process of GenIR models by applying methods based on mechanistic interpretability, such as patching and vocabulary projections. By replacing the GenIR encoder with one trained on fewer documents, we demonstrate that the decoder is the primary component responsible for successful retrieval. Our patching experiments reveal that not all components in the decoder are crucial for the retrieval process. More specifically, we find that a pass through the decoder can be divided into three stages: (I) the priming stage, which contributes important information for activating subsequent components in later layers; (II) the bridging stage, where cross-attention is primarily active to transfer query information from the encoder to the decoder; and (III) the interaction stage, where predominantly MLPs are active to predict the document identifier. Our findings indicate that interaction between query and document information occurs only in the last stage. We hope our results promote a better understanding of GenIR models and foster future research to overcome the current challenges associated with these models.

Problem

• Understanding internal computations of GenIR models
• Identifying key decoder stages for document retrieval
• Analyzing query-document interaction mechanisms in GenIR

Innovation

• Uses transformer encoder-decoder for end-to-end ranking
• Applies mechanistic interpretability methods like patching
• Identifies three key decoder stages for retrieval