Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency

2026-04-27

AI Summary

This work addresses the challenge of deploying large vision-language models (LVLMs) on edge devices due to their substantial computational and memory demands, compounded by the inflexibility and high cost of existing compression methods. The authors propose an efficient compression framework that combines structured pruning—specifically layer and width pruning—of the language backbone with lightweight recovery training using only 5% of the original data. They demonstrate that width pruning is particularly advantageous under low-resource conditions and reveal that fine-tuning only the multimodal projector suffices for significant performance recovery. By further integrating supervised fine-tuning with hidden-state knowledge distillation, the method preserves over 95% of the original performance across three LVLMs ranging from 3B to 7B parameters, substantially reducing both computational overhead and data requirements for compression.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While Large Vision Language Models (LVLMs) demonstrate impressive capabilities, their substantial computational and memory requirements pose deployment challenges on resource-constrained edge devices. Current parameter reduction techniques primarily involve training LVLMs from small language models, but these methods offer limited flexibility and remain computationally intensive. We study a complementary route: compressing existing LVLMs by applying structured pruning to the language model backbone, followed by lightweight recovery training. Specifically, we investigate two structural pruning paradigms: layerwise and widthwise pruning, and pair them with supervised finetuning and knowledge distillation on logits and hidden states. Additionally, we assess the feasibility of conducting recovery training with only a small fraction of the available data. Our results show that widthwise pruning generally maintains better performance in low-resource scenarios, where computational resources are limited or there is insufficient finetuning data. As for the recovery training, finetuning only the multimodal projector is sufficient at small compression levels. Furthermore, a combination of supervised finetuning and hidden-state distillation yields optimal recovery across various pruning levels. Notably, effective recovery can be achieved using just 5% of the original data, while retaining over 95% of the original performance. Through empirical study on three representative LVLM families ranging from 3B to 7B parameters, this study offers actionable insights for practitioners to compress LVLMs without extensive computation resources or sufficient data. The code base is available at https://github.com/YiranHuangIrene/VLMCompression.git.

Problem

• Large Vision Language Models
• Structural Pruning
• Model Compression
• Data Efficiency
• Edge Deployment

Innovation

• structured pruning
• vision-language models
• data-efficient recovery
• knowledge distillation
• widthwise pruning