QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking

2025-10-13

AI Summary

This paper addresses the limited accuracy of document re-ranking in neural information retrieval. We propose QDER, a fine-grained re-ranking model that jointly leverages knowledge graph semantics and multi-vector representations. Its core innovation is a late-aggregation mechanism that jointly models query–document matching at both token and entity granularities, enabling query-aware semantic alignment via attention-based transformation, vector decomposition, and local similarity computation—aggregating fine-grained matching signals only at the final scoring stage. To enhance semantic understanding, QDER integrates knowledge graph embeddings, significantly improving retrieval performance for complex and difficult queries. Extensive experiments on five standard benchmarks demonstrate state-of-the-art results: on TREC Robust 2004, QDER achieves a 36% relative improvement in nDCG@20 over the strongest baseline, reaching 0.70 for difficult queries. These results validate the effectiveness of knowledge-guided, multi-vector, fine-grained matching for re-ranking.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Natural Language Processing: Question Answering
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Neural IR has advanced through two distinct paths: entity-oriented approaches leveraging knowledge graphs and multi-vector models capturing fine-grained semantics. We introduce QDER, a neural re-ranking model that unifies these approaches by integrating knowledge graph semantics into a multi-vector model. QDER's key innovation lies in its modeling of query-document relationships: rather than computing similarity scores on aggregated embeddings, we maintain individual token and entity representations throughout the ranking process, performing aggregation only at the final scoring stage - an approach we call "late aggregation." We first transform these fine-grained representations through learned attention patterns, then apply carefully chosen mathematical operations for precise matches. Experiments across five standard benchmarks show that QDER achieves significant performance gains, with improvements of 36% in nDCG@20 over the strongest baseline on TREC Robust 2004 and similar improvements on other datasets. QDER particularly excels on difficult queries, achieving an nDCG@20 of 0.70 where traditional approaches fail completely (nDCG@20 = 0.0), setting a foundation for future work in entity-aware retrieval.

Problem

• Unifying entity-oriented and multi-vector neural retrieval approaches
• Integrating knowledge graph semantics into fine-grained document representations
• Improving ranking accuracy through late aggregation of token representations

Innovation

• Integrates knowledge graphs into multi-vector retrieval model
• Uses late aggregation of token and entity representations
• Applies learned attention patterns for fine-grained matching