Don't Go Breaking My LLM: The Impact of Pruning Attention Layers on Explanation Faithfulness and Confidence Calibration

2026-06-23

AI Summary

This study addresses a critical gap in large language model (LLM) compression research, which has predominantly prioritized accuracy while overlooking interpretability and reliability. The authors systematically investigate the impact of attention layer pruning on explanation faithfulness and confidence calibration, conducting empirical evaluations across five prominent LLMs and eight diverse datasets. Their findings reveal that, although pruning often preserves task accuracy, it can substantially degrade explanation faithfulness and severely impair confidence calibration. This work underscores the limitations of relying solely on accuracy as a metric for assessing compressed models and, for the first time, advocates for the integration of interpretability and calibration metrics into a comprehensive evaluation framework for model compression.

Abstract

Pruning Large Language Models (LLMs) reduces memory and inference costs by removing parts of the network, producing smaller models that retain most of their accuracy. As attention layers are the most resource-intensive parts of LLMs, pruning them is a promising compression strategy. Prior work shows that up to 33% of attention layers can be pruned with minimal accuracy loss. Nevertheless, the impact of attention pruning on model interpretability, specifically faithfulness and confidence calibration, remains unstudied. To address this gap, we study how pruning attention layers affects explanation faithfulness and confidence calibration across five LLMs and eight datasets. While the pruned models often maintain high accuracy, we find that their faithfulness and calibration often degrade. Notably, faithfulness and calibration can fluctuate significantly, even when accuracy remains stable, highlighting a misalignment between model confidence, interpretability, and accuracy. Our findings suggest that layer pruning can affect LLMs' interpretability and reliability in ways not captured by accuracy and efficiency measures alone. We recommend including explainability and calibration metrics when evaluating pruned models.

Problem

• pruning
• attention layers
• explanation faithfulness
• confidence calibration
• Large Language Models

Innovation

• attention pruning
• explanation faithfulness
• confidence calibration
• large language models
• model interpretability