ReinPool: Reinforcement Learning Pooling Multi-Vector Embeddings for Retrieval System

2026-01-12

arXiv.org

AI Summary

This work addresses the severe index inflation—exceeding a thousand-fold—caused by multi-vector embeddings, which, despite enhancing fine-grained document representation, critically undermines scalability in retrieval systems. To this end, we propose ReinPool, a novel framework that, for the first time, leverages reinforcement learning to automatically select and compress multi-vector embeddings into compact single-vector representations without requiring human annotations. The method employs a reward function grounded in NDCG and inverse retrieval objectives, enabling end-to-end training. Evaluated on the Vidore V2 benchmark, ReinPool achieves index compression by 746–1249× while recovering 76–81% of the original retrieval performance, significantly outperforming mean pooling by 22–33% in NDCG@3.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Data Compression
• Machine Learning: Representation Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Multi-vector embedding models have emerged as a powerful paradigm for document retrieval, preserving fine-grained visual and textual details through token-level representations. However, this expressiveness comes at a staggering cost: storing embeddings for every token inflates index sizes by over $1000\times$ compared to single-vector approaches, severely limiting scalability. We introduce \textbf{ReinPool}, a reinforcement learning framework that learns to dynamically filter and pool multi-vector embeddings into compact, retrieval-optimized representations. By training with an inverse retrieval objective and NDCG-based rewards, ReinPool identifies and retains only the most discriminative vectors without requiring manual importance annotations. On the Vidore V2 benchmark across three vision-language embedding models, ReinPool compresses multi-vector representations by $746$--$1249\times$ into single vectors while recovering 76--81\% of full multi-vector retrieval performance. Compared to static mean pooling baselines, ReinPool achieves 22--33\% absolute NDCG@3 improvement, demonstrating that learned selection significantly outperforms heuristic aggregation.

Problem

• multi-vector embeddings
• retrieval system
• index scalability
• embedding compression
• storage cost

Innovation

• Reinforcement Learning
• Multi-vector Embeddings
• Retrieval Optimization
• Dynamic Pooling
• Index Compression