Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions

2026-05-08

AI Summary

This study addresses the abrupt performance collapse commonly observed during layer pruning of large language models, a phenomenon whose underlying mechanism remains poorly understood. Introducing, for the first time, a decision representation perspective, the work proposes two quantitative metrics—decision boundary sharpness and option frequency—and combines them with an iterative layer pruning framework to uncover a sudden transition between silent and decision-making phases within the model. The findings reveal that pruning layers in the silent phase disrupts critical decision transitions, leading to catastrophic performance degradation, whereas pruning within the decision phase has minimal impact. This insight offers a novel mechanistic explanation for pruning-induced collapse and suggests a principled pathway for mitigating such failures through phase-aware pruning strategies.

Abstract

Layer pruning efficiently reduces Large Language Model (LLM) computational costs but often triggers sudden performance collapse. Existing representation-based analyses struggle to explain this mechanism. We propose studying pruning through decision representation. Focusing on multiple-choice tasks, we introduce two metrics, Decision Margin and Option Frequency, and an Iterative Pruning method to analyze layer-wise decision dynamics. Our findings reveal a sharp decision transition that partitions the network into two stages: a Silent Phase, where the model cannot yet predict the correct answer, and a Decisive Phase, where the correct prediction emerges. We also find that pruning the Decisive Phase has minimal impact, whereas pruning the Silent Phase triggers immediate performance collapse, highlighting its extreme sensitivity to structural changes. Therefore, we conclude that pruning-induced collapse stems from disrupting the Silent Phase, which prevents the critical decision transition from occurring.

Problem

• performance collapse
• layer pruning
• large language models
• decision representation
• Silent Phase

Innovation

• decision representation
• layer pruning
• performance collapse
• Silent Phase
• Decision Margin