Understanding the Behaviors of Environment-aware Information Retrieval

2026-06-15

AI Summary

This work addresses a critical yet overlooked limitation in existing retrieval-augmented generation (RAG) approaches: their failure to account for the varying query-style preferences of different retrievers, which hinders the generalizability of query generation strategies. The study systematically uncovers this phenomenon and introduces a retriever-aware adaptive query generation framework. By leveraging reinforcement learning, the framework trains a large language model to dynamically tailor its query formulation to the characteristics of diverse retrievers. To stabilize optimization over multi-step retrieval trajectories, the method incorporates a branch-and-rollback mechanism. Experimental results demonstrate substantial improvements in retrieval performance, underscoring the pivotal roles of model scale expansion and human guidance in developing effective, scalable retriever-aware RAG systems.

Abstract

Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.

Problem

• retriever-aware
• query formulation
• retrieval-augmented generation
• information retrieval
• retriever adaptation

Innovation

• retriever-aware
• query formulation
• reinforcement learning
• RAG
• branching rollout