KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking

2026-06-21

AI Summary

This work addresses the inefficiency and inflexibility of existing reranking models, which stem from tightly coupled joint encoding of queries and documents. To overcome these limitations, the authors propose an efficient, non-late-interaction reranking architecture that decouples query and document representations using an encoder–decoder framework, while preserving strong relevance modeling through an early cross-attention mechanism. The design further incorporates Matryoshka embedding pooling to enable flexible model compression. The resulting Nano/Small/Large model series achieves state-of-the-art performance across BEIR, MIRACL, and LMEB benchmarks. Notably, the Nano variant—containing only 0.27B parameters—matches or exceeds the effectiveness of much larger embedding models with 7–12B parameters, establishing a new efficiency–accuracy trade-off frontier on BEIR.

Abstract

As retrieval systems scale, high-quality reranking becomes increasingly important. However, most existing rerankers, whether encoder-based or decoder-based, jointly encode the query and passage, tightly coupling their computation and limiting deployment efficiency as well as flexibility. We present KaLM-Reranker-V1, a fast but not late-interaction (FBNL) reranker that decouples query and passage computation while retaining expressive relevance modeling. Built on an encoder-decoder architecture, KaLM-Reranker-V1 uses the encoder to pre-encode passages with Matryoshka embedding pooling, while the decoder models the system instruction, user instruction, and query intent; cross-attention then captures relevance between the query context and passage representations. This design makes KaLM-Reranker-V1 efficient through decoupled passage encoding, yet not late interaction, by preserving rich relevance modeling through cross-attention. We instantiate KaLM-Reranker-V1 in three sizes, Nano, Small, and Large, with 0.27B, 1B, and 4B activated parameters, respectively. Extensive experiments on BEIR, MIRACL, and LMEB demonstrate that KaLM-Reranker-V1 achieves strong reranking performance with superior efficiency. On BEIR, KaLM-Reranker-V1 achieves state-of-the-art performance, on par with strong industrial models such as the Qwen3-Reranker series; on MIRACL, despite not being extensively trained on multilingual data, KaLM-Reranker-V1 still shows excellent reranking performance. Moreover, on LMEB, reranking models demonstrate a clear advantage, with even the 0.27B Nano model remaining competitive with 7-12B embedding models.

Problem

• reranking
• query-passage encoding
• deployment efficiency
• computational coupling
• relevance modeling

Innovation

• fast but not late interaction
• decoupled encoding
• cross-attention reranking
• Matryoshka embedding
• encoder-decoder reranker