ARHN: Answer-Centric Relabeling of Hard Negatives with Open-Source LLMs for Dense Retrieval

2026-04-13

AI Summary

This work addresses the challenge that hard negative samples in neural retrieval training often contain false negatives and ambiguous negatives with partial correct answers, leading to inconsistent learning signals and degraded model performance. To mitigate this issue, the authors propose ARHN, a two-stage framework that introduces an answer-centric perspective for the first time. Leveraging open-source large language models, ARHN performs unsupervised relabeling and reranking of hard negatives, jointly optimizing the identification of false negatives and the filtering of ambiguous ones. This approach enables a scalable negative sample purification pipeline that significantly outperforms single-strategy baselines on the BEIR benchmark, thereby enhancing retrieval effectiveness.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Neural retrievers are often trained on large-scale triplet data comprising a query, a positive passage, and a set of hard negatives. In practice, hard-negative mining can introduce false negatives and other ambiguous negatives, including passages that are relevant or contain partial answers to the query. Such label noise yields inconsistent supervision and can degrade retrieval effectiveness. We propose ARHN (Answer-centric Relabeling of Hard Negatives), a two-stage framework that leverages open-source LLMs to refine hard negative samples using answer-centric relevance signals. In the first stage, for each query-passage pair, ARHN prompts the LLM to generate a passage-grounded answer snippet or to indicate that the passage does not support an answer. In the second stage, ARHN applies an LLM-based listwise ranking over the candidate set to order passages by direct answerability to the query. Passages ranked above the original positive are relabeled to additional positives. Among passages ranked below the positive, ARHN excludes any that contain an answer snippet from the negative set to avoid ambiguous supervision. We evaluated ARHN on the BEIR benchmark under three configurations: relabeling only, filtering only, and their combination. Across datasets, the combined strategy consistently improves over either step in isolation, indicating that jointly relabeling false negatives and filtering ambiguous negatives yields cleaner supervision for training neural retrieval models. By relying strictly on open-source models, ARHN establishes a cost-effective and scalable refinement pipeline suitable for large-scale training.

Problem

• hard negatives
• label noise
• dense retrieval
• false negatives
• ambiguous negatives

Innovation

• dense retrieval
• hard negative mining
• answer-centric relabeling
• open-source LLMs
• label noise reduction