SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization

2026-08-16

AI Summary

This study addresses the severe accuracy degradation in 2–3-bit quantization of large language models and the limitations of existing backpropagation-free optimizers by proposing SCHURQUANT. Leveraging Schur complement curvature for precise group-wise quadratic optimization, this method analytically eliminates sequential suffix responses and alternately optimizes quantization parameters with integer encoding. It further integrates teacher reconstruction, reference weight regularization, and closed-form scale refitting for synergistic optimization. Evaluated across eight Llama and Qwen models, SCHURQUANT achieves state-of-the-art zero-shot accuracy, outperforming the strongest baseline by 9.65 percentage points under 2-bit settings. These results demonstrate a significant breakthrough in overcoming performance bottlenecks associated with extremely low-bit quantization, establishing a new standard for efficient LLM compression without retraining.

Abstract

Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits. Existing backpropagation-free PTQ optimizers have two limitations: group decisions ignore the correction that the remaining continuous suffix can absorb, and discrete refinements typically keep the affine quantization grid fixed. We introduce SCHUROPT, which analytically eliminates the suffix's optimal continuous response, yielding an exact groupwise quadratic with Schur-complement curvature. It then alternates closed-form row-wise scale/zero-point refitting with coordinate descent over integer codes. With the GPTQ objective fixed, SCHUROPT improves mean zero-shot accuracy on 2-bit Qwen3-4B by 11.88 percentage points (pp). At higher precision, however, tighter reconstruction does not consistently improve end-model metrics. SCHURQUANT therefore combines SCHUROPT with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and teacher-decision token weighting. Across eight Llama and Qwen models, SCHURQUANT achieves the highest mean zero-shot accuracy among the evaluated backpropagation free PTQ baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.

Problem

• Post-training quantization
• Large language models
• Low-bit quantization
• Backpropagation-free optimization
• Discrete optimization

Innovation

• Schur-complement curvature
• Groupwise discrete optimization
• Backpropagation-free PTQ
• Quantized-prefix teacher reconstruction
• Coordinate descent