Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models

2026-08-21

AI Summary

研究针对量化语言模型中的不确定性保留问题，提出了一种基于目标的校准数据选择方法Doubt-Preserving Quantization (DPQ)，以适应不同部署需求。

Abstract

Quantization is widely used to deploy large language models, but its effect on uncertainty behavior, such as confidence, margins, and abstention, is rarely treated as a primary objective. We frame calibration-data selection for quantization as a target-dependent uncertainty-preservation problem. Different deployments emphasize different regions of the input distribution, yet prior work mainly optimizes accuracy-oriented compression metrics or adjusts scores after quantization. We formalize this goal with distributional and boundary preservation risks, and provide a simple mixture-mismatch argument explaining why no single calibration recipe should be expected to fit all targets. We introduce Doubt-Preserving Quantization (DPQ), a lightweight pre-quantization recipe family that uses full-precision predictions to construct target-aligned calibration mixtures of high-doubt examples and generic anchors. Across 8 language models, 9 NLP benchmarks, and 22 comparison methods, the leading fixed recipe changes with the preservation target: DPQ-r75 leads on SQuAD2 answerability-boundary preservation, while milder or single-signal variants, including DPQ-r50, confidence-only, and entropy-only, better preserve broad multiple-choice QA behavior. These results show that calibration data should be selected for the specific full-precision score behavior a deployment needs to preserve, rather than treated as a fixed quantization detail.

Problem

• Quantization
• Uncertainty Preservation
• Calibration Data Selection
• Language Models
• Deployment Specific

Innovation

• Target-Aware Calibration
• Uncertainty Preservation
• Doubt-Preserving Quantization
• Distributional Risk
• Boundary Preservation