ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training

2026-06-14

AI Summary

This work addresses the severe degradation in reasoning performance of large language models under 4-bit quantization, particularly the loss of accuracy in low-entropy symbols such as digits and operators, which existing post-training quantization (PTQ) and quantization-aware training (QAT) methods struggle to recover. The authors propose ReQAT, a novel framework that identifies low-entropy tokens during inference as quantization-sensitive points and introduces three core techniques: Trace-Aligned QAT, Selective Entropy Minimization, and Quantization-Friendly Initialization (Q-FIT), collectively optimizing critical decision points. Combined with a RoPE-consistent KV cache transformation and enhancements to the FP4 format, ReQAT achieves higher accuracy than BF16 fine-tuning under full W4A4KV4 quantization and delivers up to 3.9× and 3.1× throughput speedups on NVIDIA DGX Spark and B200 systems, respectively, within the same training budget.

Abstract

Large Reasoning Models (LRMs) achieve strong problem-solving through long chain-of-thought, but their deployment is constrained by the high cost of full-precision inference and growing KV cache footprints. Microscaled FP4 formats enable efficient FP4 deployment; however, fully quantizing weights, activations, and KV caches (W4A4KV4) causes severe reasoning degradation that existing PTQ and QAT fail to recover. We identify that FP4 failures concentrate on low-entropy tokens--precise symbolic commitments such as digits and operators--where quantization noise inflates sampling errors that cascade through reasoning traces. Based on this insight, we propose ReQAT, a reasoning-centric FP4 training framework with three components: (i) Trace-Aligned QAT (TAQ), which revisits identical reasoning traces to focus updates on critical low-entropy decisions; (ii) Selective Entropy Minimization (SEM), which reinforces confidence at low-entropy positions; and (iii) Q-FIT, a quantization-friendly initialization that jointly calibrates RoPE-consistent KV cache transformations to stabilize QAT. Under the same training budget, ReQAT not only recovers but surpasses BF16 fine-tuning accuracy, while delivering up to 3.9x throughput speedup on NVIDIA DGX Spark and 3.1x on B200.

Problem

• 4-bit quantization
• reasoning degradation
• low-entropy tokens
• KV cache quantization
• quantization-aware training

Innovation

• ReQAT
• FP4 quantization
• quantization-aware training
• low-entropy tokens
• KV cache compression