RSRank: Learning Relevance from Representational Shifts

2026-06-15

AI Summary

Existing reranking methods rely on heuristic thresholds and leverage logit signals from language models—originally designed for next-token prediction—to assess relevance, which limits their effectiveness. This work proposes a novel mechanism that evaluates relevance by analyzing the representational shift of a query induced by a document, measuring its alignment with the shift caused by an ideal (highly relevant) document. A lightweight training framework is introduced to map this alignment into well-calibrated relevance scores. By directly using representational shift alignment as a relevance criterion, the approach naturally filters out irrelevant content without requiring heuristic thresholds. Experimental results demonstrate that the proposed method significantly outperforms state-of-the-art rerankers across multiple retrieval benchmarks.

Abstract

As enterprises deploy RAG-based systems to provide grounded responses to user queries, reranking has become a critical component for the final filtering step that separates relevant from distracting or irrelevant documents. Existing rerankers often rely on heuristic thresholds to achieve optimal filtering. Moreover, for relevance scoring, state-of-the-art methods use a language model's logit signals, which are designed for next-token prediction, not for assessing relevance. To address these limitations, we identify a principled signal for relevance: the representational shift (RS) induced in a query's internal state when conditioned on a document. We observe that the alignment between (a) RS induced by a candidate document and (b) RS induced by an oracle document-set provides a robust indicator of relevance. Building on this insight, we introduce a lightweight training framework that learns projections mapping RS to calibrated relevance scores. Our training objectives naturally filter irrelevant content at a zero threshold, reducing dependence on heuristic tuning. Across diverse retrieval datasets, our method delivers gains over SOTA rerankers.

Problem

• reranking
• relevance scoring
• representational shift
• RAG systems
• heuristic thresholds

Innovation

• Representational Shift
• Reranking
• Relevance Scoring
• RAG
• Zero-threshold Filtering