REGENT: Relevance-Guided Attention for Entity-Aware Multi-Vector Neural Re-Ranking

2025-10-13

AI Summary

Neural rerankers suffer from limited performance on complex queries and long documents, primarily due to the absence of explicit modeling of key semantic units—such as entities—causing attention mechanisms to fail in focusing on salient content. To address this, we propose the Entity-Guided Reranker, the first neural reranking model to directly inject fine-grained entity semantics into the attention mechanism, using entities as a semantic scaffold for relevance-driven content selection. Our approach integrates multi-vector representations, entity-aware attention, and an end-to-end trainable architecture to jointly capture lexical matching and high-level semantic reasoning. Evaluated on three long-document reranking benchmarks, our model significantly outperforms strong baselines—including BM25 (+108%), ColBERT, and RankVicuna—establishing a new paradigm for entity-aware information retrieval.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Current neural re-rankers often struggle with complex information needs and long, content-rich documents. The fundamental issue is not computational--it is intelligent content selection: identifying what matters in lengthy, multi-faceted texts. While humans naturally anchor their understanding around key entities and concepts, neural models process text within rigid token windows, treating all interactions as equally important and missing critical semantic signals. We introduce REGENT, a neural re-ranking model that mimics human-like understanding by using entities as a "semantic skeleton" to guide attention. REGENT integrates relevance guidance directly into the attention mechanism, combining fine-grained lexical matching with high-level semantic reasoning. This relevance-guided attention enables the model to focus on conceptually important content while maintaining sensitivity to precise term matches. REGENT achieves new state-of-the-art performance in three challenging datasets, providing up to 108% improvement over BM25 and consistently outperforming strong baselines including ColBERT and RankVicuna. To our knowledge, this is the first work to successfully integrate entity semantics directly into neural attention, establishing a new paradigm for entity-aware information retrieval.

Problem

• Neural re-rankers struggle with complex information needs
• Models miss semantic signals due to rigid token processing
• Intelligent content selection is needed for lengthy documents

Innovation

• Entity-guided attention mechanism for neural re-ranking
• Integration of relevance guidance into attention computation
• Combining lexical matching with semantic reasoning