DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning

2025-09-28

AI Summary

In visual document retrieval (VDR), large vision-language models (LVLMs) adopt a multi-vector paradigm for fine-grained representation, yet require storing hundreds of patch embeddings per page—imposing prohibitive storage overhead and hindering large-scale deployment. To address this, we propose the first adaptive embedding pruning framework guided by intra-document patch attention distributions: leveraging intermediate-layer attention maps from LVLMs, our method dynamically identifies and removes redundant patch embeddings while preserving semantic fidelity. Crucially, it requires no additional training or fine-tuning and is compatible with mainstream LVLM architectures. Evaluated across 14 standard VDR benchmarks, our approach achieves 50–60% average storage compression with only a marginal mAP degradation of 0.3–0.8 percentage points. This substantially enhances deployment efficiency and scalability of LVLM-based VDR systems.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Visual Document Retrieval (VDR), the task of retrieving visually-rich document pages using queries that combine visual and textual cues, is crucial for numerous real-world applications. Recent state-of-the-art methods leverage Large Vision-Language Models (LVLMs) in a multi-vector paradigm, representing each document as patch-level embeddings to capture fine-grained details. While highly effective, this approach introduces a critical challenge: prohibitive storage overhead, as storing hundreds of vectors per page makes large-scale deployment costly and impractical. To address this, we introduce DocPruner, the first framework to employ adaptive patch-level embedding pruning for VDR to effectively reduce the storage overhead. DocPruner leverages the intra-document patch attention distribution to dynamically identify and discard redundant embeddings for each document. This adaptive mechanism enables a significant 50-60% reduction in storage for leading multi-vector VDR models with negligible degradation in document retrieval performance. Extensive experiments across more than ten representative datasets validate that DocPruner offers a robust, flexible, and effective solution for building storage-efficient, large-scale VDR systems.

Problem

• Reduces storage overhead in multi-vector visual document retrieval systems
• Addresses prohibitive storage costs from patch-level embedding methods
• Enables scalable deployment via adaptive pruning of redundant embeddings

Innovation

• Adaptive pruning reduces storage for multi-vector retrieval
• Leverages intra-document attention to discard redundant embeddings
• Achieves 50-60% storage reduction with minimal performance loss