Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning

2026-07-02

AI Summary

This work addresses the susceptibility of existing vision-language models to textual noise and visual feature fragmentation under dense instructions and fine-grained queries, primarily caused by conventional token selection strategies. To mitigate these issues, the authors propose the Entropy-Aware Dense Pruning (EADP) framework, which introduces statistical entropy to quantify and filter cross-modal textual noise, thereby generating robust instruction-relevance scores. Building on this, token selection is formulated as a submodular maximization problem incorporating spatial priors, effectively balancing semantic completeness and spatial coherence. Under strict token budgets, EADP achieves a superior accuracy-efficiency trade-off and sets new state-of-the-art results across multiple multimodal benchmarks, while preserving critical fine-grained visual information.

Abstract

Visual token pruning is a crucial strategy for accelerating VLMs by compressing redundant image patches, yet existing methods often fail to preserve critical cues under dense instructions and fine-grained queries. In this paper, we investigate this failure and identify two underlying bottlenecks: the widespread dispersion of textual noise that corrupts dense cross-modal scoring, and the feature fragmentation inherent to standard token selection. To address these issues, we propose Entropy-Aware Dense Pruning (EADP), a framework that reformulates pruning as a structured compression problem. EADP first leverages statistical entropy to quantify and filter out textual noise, yielding a robust, fine-grained instruction relevance score. Subsequently, instead of naive Top-K selection, EADP casts token selection as a submodular maximization problem with a spatial prior, explicitly ensuring a holistic and non-redundant visual representation. Extensive experiments demonstrate that EADP improves the accuracy-efficiency trade-off of VLMs, robustly preserving fine-grained visual cues under strict token budgets while achieving SoTA performance on challenging multimodal benchmarks.

Problem

• visual token pruning
• textual noise
• feature fragmentation
• dense instructions
• fine-grained queries

Innovation

• visual token pruning
• entropy-aware
• submodular maximization
• textual noise filtering
• dense instruction grounding