DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark

2026-05-28

AI Summary

This work addresses key challenges in multimodal document retrieval—namely, semantic ambiguity in dense embeddings, limited generalization of supervised rerankers, and the narrow scope of existing evaluation benchmarks—by proposing a plug-and-play retrieval framework. The framework introduces a novel OCR-free, layout-aware sparse embedding and a few-shot generalizable reranker enhanced with reasoning-augmented examples and optimized sampling, further refined through a hybrid encoding strategy to improve retrieval accuracy. To enable comprehensive evaluation, the authors also construct MultiDocR, a multidimensional benchmark covering diverse retrieval scenarios. Experimental results demonstrate that the proposed method significantly outperforms current state-of-the-art approaches across multiple benchmarks, confirming its superior accuracy and generalization capability.

Abstract

Multimodal documents contain diverse elements, such as tables, figures, and layouts, which can complicate retrieval tasks. While current approaches typically combine dense visual embedding models with supervised rerankers to achieve high-precision retrieval, they face inherent limitations. First, the coarse-grained nature of dense embeddings tends to obfuscate explicit semantics, failing to leverage structurally salient information. Second, supervised reranking models suffer from generalization bottlenecks, as their performance heavily relies on domain-specific training data. Furthermore, existing benchmarks often lack diverse assessment dimensions and comprehensive relevance annotations, limiting reliable evaluation. To address these challenges, we propose DocRetriever, a plug-and-play framework. It enhances visual retrieval via a layout-aware sparse embedding technique, enabling effective hybrid encoding without the overhead of optical character recognition (OCR). We also introduce a generalizable reranker that leverages reasoning-augmented demonstrations and optimized sampling to improve accuracy in few-shot settings. Finally, we construct a new benchmark, MultiDocR, to enable more rigorous evaluation. Experiments across diverse benchmarks validate DocRetriever's superiority over state-of-the-art methods.

Problem

• multimodal document retrieval
• dense embedding limitations
• supervised reranker generalization
• benchmark evaluation
• structural semantics

Innovation

• layout-aware sparse embedding
• plug-and-play framework
• reasoning-augmented reranker
• few-shot retrieval
• multimodal document benchmark