BlockPruner: Fine-grained Pruning for Large Language Models

2024-06-15

arXiv.org

AI Summary

Large language models (LLMs) suffer from high inference costs, and existing layer-wise pruning methods overlook fine-grained redundancy within individual layers. Method: This paper proposes a training-free, structured pruning approach that operates at the module level—specifically targeting multi-head attention (MHA) and MLP subcomponents within Transformer layers. It introduces a perplexity-based importance scoring mechanism coupled with a heuristic iterative search strategy, enabling structured, fine-grained compression without fine-tuning. Contribution/Results: The method is architecture-agnostic and validated across multiple LLMs. Compared to state-of-the-art pruning techniques, it reduces average downstream task accuracy loss by 37% and achieves up to 2.1× inference speedup. By pruning at the MHA/MLP level rather than coarser layer-level units, it significantly improves compression granularity and practical deployability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

With the rapid growth in the size and complexity of large language models (LLMs), the costs associated with their training and inference have escalated significantly. Research indicates that certain layers in LLMs harbor substantial redundancy, and pruning these layers has minimal impact on the overall performance. While various layer pruning methods have been developed based on this insight, they generally overlook the finer-grained redundancies within the layers themselves. In this paper, we delve deeper into the architecture of LLMs and demonstrate that finer-grained pruning can be achieved by targeting redundancies in multi-head attention (MHA) and multi-layer perceptron (MLP) blocks. We propose a novel, training-free structured pruning approach called BlockPruner. Unlike existing layer pruning methods, BlockPruner segments each Transformer layer into MHA and MLP blocks. It then assesses the importance of these blocks using perplexity measures and applies a heuristic search for iterative pruning. We applied BlockPruner to LLMs of various sizes and architectures and validated its performance across a wide range of downstream tasks. Experimental results show that BlockPruner achieves more granular and effective pruning compared to state-of-the-art baselines.

Problem

• Reduces redundancy in large language models (LLMs)
• Targets finer-grained pruning in MHA and MLP blocks
• Achieves granular pruning without training

Innovation

• Training-free structured pruning for LLMs
• Targets redundancies in MHA and MLP blocks
• Uses perplexity and heuristic search for pruning