Improving Document Retrieval Coherence for Semantically Equivalent Queries

2025-08-11

AI Summary

Dense retrieval (DR) models often yield inconsistent rankings for semantically equivalent but lexically distinct queries, undermining robustness and reliability. Method: We propose an improved multi-negative ranking loss that explicitly enforces distributional consistency of top-k retrieved documents across semantically equivalent queries, without introducing additional parameters or inference overhead. Contribution/Results: Our approach simultaneously improves both retrieval consistency and accuracy—the first method to achieve this dual gain. Extensive evaluation on MS-MARCO, Natural Questions, BEIR, and TREC DL demonstrates substantial robustness gains: average NDCG@10 improvements of 1.2–2.8%, and a 15.3–22.7% increase in result overlap across query variants. These results indicate significantly reduced sensitivity to lexical paraphrasing while maintaining or enhancing retrieval effectiveness. The method provides a principled, parameter-free solution for building more robust and trustworthy dense retrieval systems.

Technology Category

• Search and Optimization: Distributed Search
• Natural Language Processing: Safety and Robustness
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Dense Retrieval (DR) models have proven to be effective for Document Retrieval and Information Grounding tasks. Usually, these models are trained and optimized for improving the relevance of top-ranked documents for a given query. Previous work has shown that popular DR models are sensitive to the query and document lexicon: small variations of it may lead to a significant difference in the set of retrieved documents. In this paper, we propose a variation of the Multi-Negative Ranking loss for training DR that improves the coherence of models in retrieving the same documents with respect to semantically similar queries. The loss penalizes discrepancies between the top-k ranked documents retrieved for diverse but semantic equivalent queries. We conducted extensive experiments on various datasets, MS-MARCO, Natural Questions, BEIR, and TREC DL 19/20. The results show that (i) models optimizes by our loss are subject to lower sensitivity, and, (ii) interestingly, higher accuracy.

Problem

• Improves coherence in document retrieval for semantically similar queries
• Reduces sensitivity of Dense Retrieval models to query variations
• Enhances accuracy and consistency in top-k document rankings

Innovation

• Multi-Negative Ranking loss variation
• Trains Dense Retrieval models
• Improves coherence for semantic queries