MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving

2026-06-25

AI Summary

This work addresses the computational inefficiency of multi-view vision-language models (VLMs) in autonomous driving, which stems from excessively long visual token sequences and is exacerbated by existing pruning methods that overlook dynamic inter-view importance variations and shifting information demands during inference. To tackle this, the authors propose MVPruner, a two-stage adaptive pruning framework that first allocates pruning budgets based on view-wise information diversity while preserving cross-stage consistent tokens, and then performs instruction-guided, task-aligned pruning. The study reveals for the first time that deep-layer encoding in multi-view VLMs establishes task-specific view priors and exhibits dynamic information needs, enabling the design of a dynamic pruning mechanism synchronized with the reasoning process. Evaluated on four benchmarks including DriveLM, MVPruner achieves substantial efficiency gains—e.g., on DriveMM, it reduces FLOPs by 87.3%, accelerates prefilling by 4.97×, and retains 98.5% of the original accuracy.

Abstract

Vision-Language Models (VLMs) improve generalization and interpretability in autonomous driving but suffer from efficiency issues due to long visual token sequences, particularly in standard multi-view settings. Existing token pruning methods employ fixed pruning rate allocation and static importance metrics, ignoring dynamic inter-view importance differences and the evolving information importance during inference. Our analysis reveals that multi-view VLMs inherently encode task-related view priors in deeper layers and exhibit dynamic information requirements. Motivated by these findings, we propose MVPruner, a two-stage adaptive token pruning method that aligns pruning behavior with the model's dynamic information requirements. The first stage allocates pruning budgets based on the information diversity of each view, and retains tokens with consistent contribution across stages, ensuring semantic representational capacity. The second stage allocates budgets and selects tokens guided by instruction text to guarantee task alignment. Experimental results on four benchmarks demonstrate the superior performance of our method. For example, DriveMM equipped with MVPruner achieves 87.3% reduction in FLOPs, 4.97* speedup in prefilling phase while retaining 98.5% accuracy on DriveLM benchmark.

Problem

• token pruning
• multi-view vision-language models
• autonomous driving
• efficiency
• dynamic importance

Innovation

• dynamic token pruning
• multi-view vision-language models
• adaptive pruning
• instruction-guided pruning
• autonomous driving