Dual-View Training for Instruction-Following Information Retrieval

2026-04-20

AI Summary

This work addresses the challenge that existing retrieval systems often conflate semantic relevance with user-specified instruction constraints, frequently misranking thematically related but instruction-violating documents as relevant. To mitigate this, the authors propose a dual-perspective data synthesis method based on instruction polarity inversion: for each query-document pair, a large language model generates complementary instructions that invert the relevance label, compelling the retriever to dynamically adjust its judgments over the same candidate set according to differing instructions. This approach significantly enhances the model’s sensitivity to explicit instructions rather than reliance on thematic cues alone. A 305M-parameter dense retrieval encoder trained with this strategy achieves a 45% performance gain on the FollowIR benchmark, outperforming general-purpose embedding models of comparable or larger scale, thereby underscoring the critical role of instruction-aware supervision and data diversity in improving instruction-following retrieval capabilities.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Instruction-following information retrieval (IF-IR) studies retrieval systems that must not only find documents relevant to a query, but also obey explicit user constraints such as required attributes, exclusions, or output preferences. However, most retrievers are trained primarily for semantic relevance and often fail to distinguish documents that match the topic from those that satisfy the instruction. We propose a dual-view data synthesis strategy based on polarity reversal: given a query, a document that is relevant under the instruction, and a hard negative that matches the query but violates the instruction, we prompt an LLM to generate a complementary instruction under which the two documents swap relevance labels. By presenting the same document pair under complementary instructions that invert their relevance labels, the training signal forces the retriever to reconsider the same candidate set through the instruction, rather than relying on fixed topical cues. On a 305M-parameter encoder, our method improves performance on the FollowIR benchmark by 45%, surpassing general-purpose embedding models of comparable or larger scale. Through head-to-head comparisons at matched data budgets, we further show that data diversity and instruction supervision play complementary roles: the former preserves general retrieval quality, while the latter improves instruction sensitivity. These results highlight the value of targeted data synthesis for building retrieval systems that are both broadly capable and instruction-aware.

Problem

• instruction-following information retrieval
• relevance modeling
• user constraints
• instruction sensitivity
• document ranking

Innovation

• dual-view training
• instruction-following retrieval
• polarity reversal
• data synthesis
• retriever fine-tuning