Breaking the Rounding Trap: Securing LLMs against Quantization-Conditioned Backdoors

2026-06-28

AI Summary

This work addresses the threat of quantization-aware backdoor attacks, which exploit rounding errors to activate malicious behavior post-quantization while evading conventional detection. The authors propose QuantGuard, a defense mechanism that requires no modification to existing quantization algorithms and operates with minimal calibration data. By introducing differentiable rounding control variables, QuantGuard precisely manipulates critical rounding decisions through three key components: error-guided rounding inversion, output distribution consistency constraints, and weight distance regularization. This approach disrupts the attacker’s carefully aligned weight–quantization boundary, effectively neutralizing the backdoor. Evaluated across six prominent large language models and three quantization precisions, QuantGuard reduces attack success rates to levels comparable to those of clean models, incurs negligible degradation in general performance, and maintains low computational overhead.

Abstract

Model quantization is a key technique for reducing storage and inference costs when deploying large language models in practice. However, recent studies show that the discretization and rounding errors introduced by quantization can be exploited by adversaries to construct quantization-conditioned backdoor (QCB) attacks. Under such attacks, malicious behaviors remain dormant in the full-precision stage and are activated only after quantized deployment, thereby bypassing conventional security auditing and detection mechanisms. To address this threat, we propose a proactive pre-quantization defense method, QuantGuard. Our method introduces differentiable rounding control variables and combines error-guided rounding reversal constraints, output-distribution consistency, and weight-distance regularization to finely regulate critical rounding behaviors. Crucially, QuantGuard utilizes only a small calibration dataset and does not modify existing quantization algorithms. This design breaks the precise alignment between attacker-crafted weight patterns and quantization boundaries, effectively suppressing the post-quantization backdoor activation pathway while preserving the model's original functionality and performance. We conduct systematic experiments on six mainstream LLMs (including the LLaMA-3 and Qwen2.5-Coder) using three quantization precisions (INT8, FP4, and NF4) across three representative scenarios: vulnerable code generation, content injection, and over-refusal. The results show that QuantGuard consistently mitigates QCB attacks, reducing the attack success rate to a level comparable to the clean model while largely preserving performance on general capability benchmarks. With low computational overhead, our method offers an effective, practical solution for secure quantized LLM deployment.

Problem

• quantization-conditioned backdoor
• model quantization
• large language models
• security auditing
• rounding errors

Innovation

• quantization-conditioned backdoor
• differentiable rounding
• error-guided reversal
• output-distribution consistency
• weight-distance regularization