DocPrune:Efficient Document Question Answering via Background, Question, and Comprehension-aware Token Pruning

2026-04-24

AI Summary

This work addresses the inefficiency of existing vision-language models in long-document question answering, where redundant context and sparse evidence lead to excessive computation, and generic token compression methods fail to exploit document structure. The authors propose a training-free, progressive token pruning framework that dynamically retains critical tokens by jointly considering background awareness, question relevance, and the model’s layer-wise comprehension state, while adaptively selecting the optimal layer to initiate pruning. Evaluated on M3DocRAG, the method achieves a 3.0× speedup in encoder throughput and a 3.3× improvement in decoder throughput, alongside a 1.0-point gain in F1 score, significantly enhancing both efficiency and accuracy without any additional training.

Technology Category

• Computer Vision: Language and Vision
• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recent advances in vision-language models have demonstrated remarkable performance across diverse multi-modal tasks, including document question answering that leverages structured visual cues from text, tables, and figures. However, unlike natural images, document images contain large backgrounds and only sparse supporting evidence, leading to the inefficient consumption of substantial computational resources, especially for long documents. We observe that existing token-reduction methods for natural images and videos fall short in utilizing the structural sparsity unique to documents. To address this, we propose DocPrune, a training-free and progressive document token pruning framework designed for efficient long-document understanding. The proposed method preserves only the essential tokens for the task while removing unnecessary ones, such as background or question-irrelevant tokens. Moreover, it automatically selects the appropriate layers to initiate token pruning based on the model's level of comprehension. Our experiments on the M3DocRAG show that DocPrune improves throughput by 3.0x and 3.3x in the encoder and decoder, respectively, while boosting the F1 score by +1.0, achieving both higher accuracy and efficiency without any additional training.

Problem

• document question answering
• token pruning
• computational efficiency
• structural sparsity
• long-document understanding

Innovation

• token pruning
• document question answering
• vision-language models
• computational efficiency
• training-free