Test-Time Compute for Dense Retrieval: Agentic Program Generation with Frozen Embedding Models

2026-05-11

AI Summary

This work addresses the challenge of enhancing retrieval performance of frozen small-scale embedding models without retraining. It proposes a test-time computation approach that leverages agent-based program search to automatically generate and evaluate inference programs for optimizing dense retrieval. The key insight is the discovery of a parameter-agnostic, unified algebraic formulation: interpolating the original query vector with the softmax-weighted centroid of its local top-K retrieved documents. This method consistently improves nDCG@10 across seven embedding model families spanning an order-of-magnitude difference in parameter count and demonstrates broad effectiveness on the full BEIR benchmark suite. To the best of our knowledge, this is the first demonstration that test-time computation can substantially boost the retrieval capability of frozen embedding models.

Abstract

Test-time compute is widely believed to benefit only large reasoning models. We show it also helps small embedding models. Most modern embedding checkpoints are distilled from large LLM backbones and inherit their representation space; a frozen embedding model should therefore benefit from extra inference compute without retraining. Using an agentic program-search loop, we explore 259 candidate inference programs over a frozen embedding API across ninety generations. The entire Pareto frontier collapses onto a single algebra: a softmax-weighted centroid of the local top-K documents interpolated with the query. This parameter-free default lifts nDCG@10 statistically significantly across seven embedding-model families spanning a tenfold parameter range, with held-out full-BEIR validation confirming the lift on every model tested.

Problem

• test-time compute
• dense retrieval
• embedding models
• frozen models
• inference optimization

Innovation

• test-time compute
• dense retrieval
• frozen embedding models
• agentic program generation
• parameter-free inference