When Reranking Hurts: Uncertainty-Based Gating for Few-Shot Reranking

2026-06-29

AI Summary

This work challenges the prevailing assumption in few-shot reranking that reranking invariably improves performance, highlighting instead that its substantial computational overhead can lead to net negative gains. The study is the first to demonstrate that reranking is not universally beneficial and introduces a training-free gating mechanism that leverages the input uncertainty estimated by large language models as a gating signal to dynamically decide whether to apply reranking. By decoupling uncertainty estimation from the retrieval-and-selection pipeline, the proposed approach achieves both generality and efficiency. Extensive experiments across eight large language models, seven natural language understanding datasets, and nine machine translation domain combinations show that the method reduces computational costs by 15%–80% compared to baselines while achieving up to a 2% performance improvement.

Abstract

Few-shot selection typically assumes that reranking retrieved examples always improves performance. We challenge this view by identifying that the expensive reranking step can in fact degrade performance. Instead, we propose \emph{Training-Free Gated Reranking}, which decides whether to rerank the few-shot examples based on the model's uncertainty. Extensive experiments across 8 LLMs, covering 7 NLU datasets and 9 MT domain-language combinations, demonstrate that our approach reduces computational costs by 15\%-80\% while improving average performance by up to 2\%. These findings indicate that higher computational cost does not guarantee better performance, and that reranking is most beneficial when targeted at high-uncertainty instances.

Problem

• few-shot reranking
• performance degradation
• uncertainty
• computational cost
• retrieved examples

Innovation

• uncertainty-based gating
• few-shot reranking
• training-free
• computational efficiency
• large language models