MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs

2026-06-30

AI Summary

Existing single-scale visual resampling methods struggle to simultaneously preserve fine-grained local details and capture global context under a fixed token budget, often leading to loss of critical visual information. To address this limitation, this work proposes MS-Resampler, a novel multi-scale visual resampling framework that introduces, for the first time, a multi-branch resampling mechanism. By explicitly injecting spatial-range priors and adaptively fusing representations across multiple scales, MS-Resampler generates superior visual features with negligible additional computational overhead. Extensive experiments demonstrate that the proposed method significantly outperforms current single-scale approaches across ten public multimodal benchmarks, consistently enhancing both visual understanding and multimodal reasoning capabilities.

Abstract

Multimodal large language models (MLLMs) typically employ resampling-based projectors to transform dense visual features into a compact token sequence for language modeling. Most existing resamplers adopt a single, fixed aggregation scope via global cross-attention, which can blur fine-grained local evidence and limit the ability to capture both local details and global context within a fixed token budget. In this work, we propose MS-Resampler, a multi-scope visual resampling framework for MLLMs. MS-Resampler instantiates multiple scope-specific resamplers by injecting explicit spatial scope priors into the resampling attention, enabling each branch to aggregate visual information at a particular granularity from local to global. The outputs of these scope-specific resamplers are then adaptively fused to produce the final visual representations for language modeling. Extensive experiments on ten public multimodal benchmarks show that MS-Resampler consistently improves visual understanding and multimodal reasoning over conventional single-scope resamplers, while introducing only minimal computational overhead.

Problem

• multimodal large language models
• visual resampling
• aggregation scope
• local details
• global context

Innovation

• multi-scope resampling
• visual feature aggregation
• spatial scope prior
• multimodal LLMs
• adaptive fusion