chroma

Multi-vector visual retrievers (e.g., ColPali-style late interaction models) deliver strong accuracy, but scale poorly because each page yields thousands of vectors, making indexing and search increasingly expensive. We present Visual RAG Toolkit, a practical system for scaling visual multi-vector retrieval with training-free, model-aware pooling and multi-stage retrieval. Motivated by Matryoshka Embeddings, our method performs static spatial pooling - including a lightweight sliding-window averaging variant - over patch embeddings to produce compact tile-level and global representations for fast candidate generation, followed by exact MaxSim reranking using full multi-vector embeddings. Our design yields a quadratic reduction in vector-to-vector comparisons by reducing stored vectors per page from thousands to dozens, notably without requiring post-training, adapters, or distillation. Across experiments with interaction-style models such as ColPali and ColSmol-500M, we observe that over the limited ViDoRe v2 benchmark corpus 2-stage retrieval typically preserves NDCG and Recall @ 5/10 with minimal degradation, while substantially improving throughput (approximately 4x QPS); with sensitivity mainly at very large k. The toolkit additionally provides robust preprocessing - high resolution PDF to image conversion, optional margin/empty-region cropping and token hygiene (indexing only visual tokens) - and a reproducible evaluation pipeline, enabling rapid exploration of two-, three-, and cascaded retrieval variants. By emphasizing efficiency at common cutoffs (e.g., k<= 10), the toolkit lowers hardware barriers and makes state-of-the-art visual retrieval more accessible in practice.

This work addresses the challenges of poor portability, high maintenance costs, and vendor lock-in arising from API fragmentation in the vector database ecosystem by proposing the first unified middleware abstraction layer for heterogeneous vector databases. The layer provides a high-level, consistent API that encapsulates core operations and employs pluggable adapters to dynamically translate requests into the native protocols of diverse backends. This approach not only enables seamless interoperability across disparate systems but also lays the groundwork for advanced query optimization at higher layers. Crucially, the design preserves full functional completeness while introducing minimal performance overhead, thereby offering a practical and efficient solution to the growing complexity of managing multiple vector database implementations.

To address the high memory overhead and low retrieval efficiency of high-dimensional vector embeddings in RAG systems, this work pioneers the full integration of 4-bit uniform quantization across the entire RAG pipeline—including FP32-to-INT4 embedding encoding/decoding, vector re-ranking calibration, and approximate nearest neighbor (ANN) search adaptation. Our method preserves semantic representation fidelity while achieving an 8× compression ratio and over 98% of the original recall rate, reducing memory footprint to 1/8 and cutting end-to-end retrieval latency by 40%. Unlike existing low-bit quantization approaches, our solution overcomes the long-standing accuracy-efficiency trade-off in semantic vector retrieval, enabling real-time deployment of million-scale document RAG on a single GPU. This work establishes a practical, production-ready pathway toward lightweight, scalable RAG systems.

This study addresses the cross-modal retrieval challenge for multimodal educational content—particularly computer science textbooks containing interleaved text and figures. We propose a multi-vector representation method that jointly encodes textual and visual semantics using a vision-language model (VLM), generating fine-grained multimodal embeddings and indexing them in a vector database to enable efficient cross-modal retrieval. Evaluated on over 3,600 pages of textbook material, we systematically compare four similarity metrics and find cosine similarity significantly outperforms alternatives. Benchmarking against 75 natural-language queries confirms substantial improvements in retrieval precision and practical utility within digital library settings. Our approach delivers a reproducible, scalable technical framework for intelligent discovery of multimodal educational resources, advancing the state of cross-modal semantic search in academic and pedagogical contexts.

Existing document retrieval systems rely on OCR followed by text embedding, resulting in lengthy pipelines, susceptibility to noise, and the neglect of critical visual cues—limiting performance in applications such as RAG. This paper proposes an image-native document retrieval paradigm that bypasses OCR entirely and performs end-to-end embedding directly on document page images. To this end, we introduce ColPali, a lightweight, trainable multimodal vision-language model integrating late-interaction matching with efficient image encoding. We further construct ViDoRe—the first benchmark dedicated to visual document retrieval. Experiments demonstrate that our approach significantly outperforms state-of-the-art methods on ViDoRe, achieves over 10× faster inference, and supports zero-shot cross-lingual and cross-domain page-level retrieval. All code, models, and data are publicly released.

This work addresses data staleness, tenant leakage, and combinatorial query explosion in production-grade retrieval-augmented generation (RAG) systems caused by decoupled data layers. To resolve these issues, the authors propose a unified data layer architecture built on PostgreSQL that, for the first time, integrates vector retrieval and structured filtering within a single database. By leveraging pgvector with HNSW indexing and a hybrid hierarchical design, the system eliminates cross-system synchronization overhead while guaranteeing strict tenant isolation and strong data consistency. Experimental results on a dataset of 50,000 documents demonstrate a 92% reduction in latency for date-filtered queries and a 74% reduction for tenant-scoped queries, alongside a 93% decrease in synchronization code, achieving zero data inconsistency and enabling efficient, secure RAG.

This work addresses the high memory overhead and poor scalability of multi-vector representations in multimodal document retrieval. To overcome these limitations, the authors propose Stellar, a novel framework that leverages high-quality lexical representations for efficient candidate pre-filtering and integrates a balanced clustering–based disk embedding storage layout with a dynamic loading strategy to enable late interaction retrieval. The approach achieves state-of-the-art retrieval effectiveness while reducing memory consumption and query latency by one to two orders of magnitude across four established benchmarks and a newly introduced large-scale dataset, substantially enhancing deployability in real-world systems.

In visual financial document retrieval, single-vector aggregation is prone to being dominated by global texture patterns, thereby obscuring critical numerical discrepancies and erroneously assigning high similarity to semantically distinct documents. This work introduces the first diagnostic benchmark for financial documents to systematically evaluate the prevalence of this issue across diverse visual encoders, embedding strategies, and aggregation methods. Experimental results demonstrate that patch-level representations effectively capture subtle semantic variations, whereas single-vector aggregation poses significant risks. The study exposes the fragility of current vision-based retrieval-augmented generation (RAG) pipelines and provides empirical evidence and actionable directions for improving document-level visual retrieval in financial contexts.

This work addresses the limitations of single-vector dense retrievers in visual document retrieval, which suffer from performance degradation due to excessive information compression, while late-interaction models incur high computational and storage costs. The authors propose a lightweight plug-in module that, without modifying the backbone network, identifies retrieval-relevant dimensions through hierarchical probing and integrates multi-layer internal representations from multimodal Transformers via a neuron-level adaptive sparse fusion mechanism to produce high-quality single-vector embeddings. Evaluated on the ViDoRe V1/V2/V3 benchmarks, the method significantly outperforms existing single-vector approaches, achieving up to a 4.5% improvement in nDCG@5 and narrowing the performance gap with late-interaction models to within 0.2.