SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization

2026-05-12

AI Summary

This work addresses the significant accuracy degradation in large language models caused by NVFP4 quantization, which stems from inflexible scale selection and the coupling of quantization and dequantization scales. To overcome this limitation, we propose SOAR, a novel framework that, for the first time, decouples and jointly optimizes these scales within the NVFP4 format. Specifically, SOAR introduces Closed-form Joint Scale Optimization (CJSO) to derive an analytical solution that minimizes reconstruction error, complemented by Discrete Scale Search (DSS), a discrete strategy that mitigates accuracy loss induced by scale quantization. Extensive experiments demonstrate that SOAR consistently outperforms existing NVFP4 methods across multiple large language models, achieving higher accuracy under identical memory constraints without requiring additional hardware support.

Abstract

NVFP4 has recently emerged as an efficient 4-bit microscaling format for large language models (LLMs), offering superior numerical fidelity with native hardware support. However, existing methods often yield suboptimal performance due to inflexible scale selection and the coupled treatment of quantization and dequantization scales. To address these issues, we propose Scale Optimization for Accurate Reconstruction (SOAR), a novel post-training quantization framework that improves the accuracy of NVFP4 quantization. At its core, SOAR features Closed-form Joint Scale Optimization (CJSO), which jointly optimizes global and block-wise scales via analytical solutions derived from reconstruction error minimization. Furthermore, it incorporates Decoupled Scale Search (DSS). DSS decouples the high-precision quantization scale from its constrained dequantization counterpart, and performs discrete search to mitigate precision loss from scale quantization. Extensive experiments across multiple LLMs show that our method consistently outperforms existing NVFP4 quantization baselines, achieving superior accuracy under the same memory footprint with no additional hardware overhead. The code and models will be available at https://github.com/steven-bao1/SOAR.

Problem

• NVFP4 quantization
• scale optimization
• post-training quantization
• reconstruction error
• large language models

Innovation

• NVFP4 quantization
• scale optimization
• post-training quantization
• reconstruction error minimization
• decoupled scale search