TASR: Training-Free Adaptive Stopping for Iterative Retrieval

2026-06-11

AI Summary

This work addresses the inefficiency in iterative retrieval-augmented generation (RAG), where redundant retrievals of already converged answers waste computational resources. The authors propose a training-free, adaptive stopping criterion that halts retrieval when the model’s normalized outputs stabilize and the order-preserving calibrated logit gap exceeds a predefined threshold. This approach constitutes the first zero-shot stopping strategy that generalizes across diverse models, tasks, and retrievers without requiring auxiliary classifiers or value networks. Evaluated across 24 experimental configurations, the method achieves 94.8% of the F1 performance of a fixed k=5 retrieval baseline while using only 62.6% of the retrieval calls, and substantially outperforms the fixed k=3 strategy.

Abstract

Iterative retrieval-augmented generation agents commonly overspend by continuing to retrieve after the model has converged on an answer, incurring calls that change neither the prediction nor the supporting evidence. Existing remedies learn a stopping policy from labeled trajectories, tying the decision to a trained component that requires retraining for each new model or task. We propose TASR (Training-Free Adaptive Stopping Rule), a one-line predicate that fires when the model repeats its previous-round normalized answer and the isotonically calibrated logit margin exceeds 0.25. No classifier or value head is learned; the threshold is fixed across all twenty-four (model, retriever, corpus) configurations we evaluate. On a 3-model x 2-dataset distractor grid, TASR retains 94.8% of fixed-k=5's macro F1 at 62.6% of its calls and exceeds fixed-k=3 by +3.42 F1. The pattern holds on nine open-domain BM25 cells (55.01 F1 at 2.98 calls vs. 54.33 at 3.00 for fixed-k=3) and, with calibration locked from the distractor split, on nine dense-retrieval cells across two retriever families, with zero significant regressions in either extension. The rule was selected from an exhaustive enumeration of 381 candidate stopping rules; no alternative Pareto-dominates it on any evaluated configuration. A signal-quality analysis shows that verbalized 1-5 confidence collapses on RLHF-tuned models (96.5% of values equal 5, entropy 0.182 nats), while the logit margin achieves 44x better class-conditional separation, grounding the design in a measurable model pathology. TASR is an auditable, training-free Pareto baseline against which learned stopping controllers can be compared. Code is publicly available.

Problem

• iterative retrieval
• stopping policy
• retrieval-augmented generation
• training-free
• adaptive stopping

Innovation

• training-free
• adaptive stopping
• retrieval-augmented generation
• logit margin
• Pareto-optimal