🤖 AI Summary
This work addresses the performance degradation in deep neural networks caused by error accumulation in conventional block-wise post-training quantization (PTQ), where a single forward pass prevents correction of early quantization errors. To mitigate this issue, the authors propose Interleaved Cross-Block Quantization (ICBQ), which refines local reconstruction by doubly optimizing adjacent “seam” block pairs at block boundaries. Built upon standard block-wise PTQ, ICBQ incorporates cross-block reconstruction within a sliding window, leveraging local shrinkage and smoothness assumptions, and is compatible with GPTQ scheduling. Experimental results demonstrate that ICBQ significantly reduces perplexity under ternary, 2-bit, and 3-bit quantization, effectively preventing performance collapse at extreme compression ratios while seamlessly integrating into existing PTQ pipelines.
📝 Abstract
Compressing large language models to two bits or fewer is increasingly feasible through block-wise post-training quantization; cross-block variants reconstruct neighboring Transformer blocks within a moving window. In the fixed two-block setting studied here, the matched sequential baseline moves this window through the network once, so errors introduced early in the sweep are not revisited. We propose Interleaved Cross-Block Quantization (ICBQ), a scheduling modification that revisits the boundary pair between consecutive chunks. Each seam pair is refined twice: first at the end of one chunk and again at the start of the next. The method retains the local two-block objective and reuses the calibration inputs of existing block-wise PTQ pipelines. Under stated local contraction and smoothness assumptions, we derive a depth-wise upper-bound comparison in which seam revisits multiply the propagated term while the residual remains bounded independently of depth. In the reported experiments, ICBQ reduces ternary-quantization perplexity relative to the matched Sequential CBQ baseline, yields finite perplexity in configurations where the baseline has severe degradation, and can also be used with 3-bit and 2-bit GPTQ.