LLM-QE: Improving Query Expansion by Aligning Large Language Models with Ranking Preferences

2025-02-24

AI Summary

To address semantic hallucinations in large language models (LLMs) during query expansion—leading to generated terms that are semantically inconsistent with the target documents—this paper proposes LLM-QE, a reinforcement learning framework that jointly optimizes two complementary reward signals: rank-based preference (aligning with dense retriever outputs) and answer-based relevance (grounded in document facts). By conditioning LLM generation on retrieved documents and integrating dual rewards, LLM-QE enforces consistency between expanded queries, the Contriever dense retriever, and factual document content. This work introduces the first document-conditioned, dual-reward-driven query expansion paradigm, effectively mitigating hallucination while enhancing both relevance and conciseness of expanded terms. Experiments demonstrate that LLM-QE improves mean average precision (mAP) by over 8% in zero-shot dense retrieval and boosts downstream retriever fine-tuning performance by more than 5%.

Technology Category

• Natural Language Processing: Question Answering
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Query expansion plays a crucial role in information retrieval, which aims to bridge the semantic gap between queries and documents to improve matching performance. This paper introduces LLM-QE, a novel approach that leverages Large Language Models (LLMs) to generate document-based query expansions, thereby enhancing dense retrieval models. Unlike traditional methods, LLM-QE designs both rank-based and answer-based rewards and uses these reward models to optimize LLMs to align with the ranking preferences of both retrievers and LLMs, thus mitigating the hallucination of LLMs during query expansion. Our experiments on the zero-shot dense retrieval model, Contriever, demonstrate the effectiveness of LLM-QE, achieving an improvement of over 8%. Furthermore, by incorporating answer-based reward modeling, LLM-QE generates more relevant and precise information related to the documents, rather than simply producing redundant tokens to maximize rank-based rewards. Notably, LLM-QE also improves the training process of dense retrievers, achieving a more than 5% improvement after fine-tuning. All codes are available at https://github.com/NEUIR/LLM-QE.

Problem

• Enhances query expansion using Large Language Models.
• Aligns LLMs with ranking preferences for better retrieval.
• Reduces LLM hallucinations during query expansion process.

Innovation

• Leverages Large Language Models
• Uses rank-based rewards
• Incorporates answer-based reward modeling