Adaptive Re-Ranking

2026-06-23

AI Summary

This work addresses the inefficiency of traditional retrieval systems that uniformly apply high-cost reranking models to all queries, incurring unnecessary latency and computational overhead for simple queries. The authors propose a utility-based adaptive reranking framework that dynamically selects reranking strategies according to query complexity, enabling cost-aware query routing. A novel utility function is introduced to guide routing decisions, and the approach leverages BM25 for sparse retrieval, MiniLM-L6-v2 for lightweight dense reranking, and BGE-v2-m3 for heavyweight neural reranking. A trained routing classifier enables multi-tier reranking strategy selection. Compared to applying the full BGE model universally, the proposed method reduces median latency by 1.15× to 53× and average latency by 1.11× to 5.22×, with nDCG@10 varying between –17.5% and +4.0%, demonstrating competitive effectiveness across multiple datasets.

Abstract

Modern Information Retrieval (IR) systems typically use a "retrieve-then-rerank" pipeline, where a computationally expensive, pre-determined cross-encoder re-ranks the top results from a fast initial retriever. While effective, this approach often applies heavy re-ranking models regardless of query complexity, resulting in high latency and wasted computational resources on simple queries. We propose Adaptive Re-Ranking, an utility-based labeling framework for cost-aware routing and present empirical evidence (via oracle analysis and a trained baseline router) that per-query routing offers large potential gains but is non-trivial to learn from limited supervision. We train a routing classifier with 3 strategies: sparse retrieval (BM25), dense re-ranking (MiniLM-L6-v2), and heavy neural re-ranking (BGE-v2-m3). Compared to BGE our method achieves 1.15-53x lower median latency and 1.11-5.22x lower mean latency across all datasets we have tested, while delivering -17.5% to +4.0% nDCG@10, which is competitive in some datasets. Our findings show that routing queries based on our novel utility function offers a scalable solution for reducing computational costs and latency in a variety of IR systems.

Problem

• Information Retrieval
• Re-Ranking
• Query Complexity
• Latency
• Computational Cost

Innovation

• Adaptive Re-Ranking
• cost-aware routing
• utility-based labeling
• query-dependent inference
• efficient information retrieval