The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis

2026-06-28

AI Summary

Current safety evaluations of large language models often overlook the interaction between quantization and sampling temperature. This study systematically assesses the safety performance of nine instruction-tuned models across FP16, INT8, and INT4 precisions under six temperature settings through factorial experiments, revealing for the first time a non-additive interaction effect between these two factors. The findings indicate that standard INT4 and INT8 quantization have largely neutral impacts on safety, whereas high temperatures substantially exacerbate decision instability in vulnerable models, with attack failure rates reaching up to 53.0%; however, no systematic cumulative effect between quantization and temperature is observed. Building on these insights, the work introduces a composite degradation index to quantify the joint degradation effect, offering a novel perspective for safety alignment during model deployment.

Abstract

Modern LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition, yet safety evaluations usually treat these choices as fixed implementation details. This leaves a practical uncertainty: does a model that is safe at FP16 and greedy decoding remain safe after it is quantized and sampled stochastically, or do the two deployment knobs amplify one another? We study this question with a factorial evaluation of 9 instruction-tuned models from six families, 3 precisions (FP16, GPTQ INT8, AWQ INT4), and 6 temperatures ($T{=}0$ to $1.0$), yielding 161 configurations and $\approx$322k responses judged by a six-model safety ensemble. Contrary to the concern that low-bit deployment broadly erodes alignment, standard non-adversarial quantization is usually safety-neutral: INT4 keeps or lowers attack success for 7 of 9 models, with clear degradation concentrated in the weakest baseline model, SmolLM3-3B ($18.5\%{\to}36.0\%$). The larger risk comes from sampling: higher temperature sharply increases decision instability for vulnerable models, with DFR reaching 53.0\% at $T{=}1.0$, even when average ASR changes modestly. Finally, the interaction is not a ``double penalty'': our Compound Degradation Index remains largely sub-additive ($-0.195$ to $+0.045$), indicating that quantization and temperature do not systematically compound. These results suggest a deployment rule of thumb: standard INT4/INT8 quantization can be reasonable for strongly aligned models, but safety claims at elevated temperature should report multi-sample stability, not only average attack success.

Problem

• quantization
• sampling temperature
• LLM safety
• alignment
• deployment

Innovation

• quantization
• sampling temperature
• safety alignment
• factorial analysis
• decision instability