An Empirical Study of OpenPangu Quantization on Ascend NPUs

2026-06-19

AI Summary

This study addresses the lack of systematic evaluation of post-training quantization robustness for the OpenPangu large language model on Ascend NPUs. On the Ascend 910B1 platform, we establish the first quantization accuracy map tailored for Ascend NPUs by uniformly calibrating and evaluating seven quantization methods—RTN, GPTQ, AWQ, SmoothQuant, GPTAQ, BiLLM, and SliM-LLM—across 18 diverse tasks. Our results demonstrate that 8-bit weight-only quantization incurs nearly no accuracy loss, while 4-bit quantization remains effective for the 7B model but causes significant degradation in the 1B model on reasoning, mathematical, and code-related tasks. Quantization to 2 bits or binary representations generally fails, with W4A4 SmoothQuant even yielding non-finite perplexity, thereby revealing the practical failure boundary of ultra-low-bit quantization on this hardware platform.

Abstract

OpenPangu models are attractive targets for private and domestic large-language-model deployment, yet their robustness under aggressive post-training quantization on Ascend NPUs has not been systematically characterized. This paper conducts a controlled empirical study of OpenPangu 1B and 7B models on Huawei Ascend 910B1 NPUs. We evaluate representative weight-only and weight-activation post-training quantization methods, including RTN, GPTQ, AWQ, SmoothQuant, GPTAQ, BiLLM, and SliM-LLM, under a unified calibration and evaluation protocol. Across 18 evaluation tasks, we find that 8-bit weight-only quantization is effectively lossless for both models, while 4-bit quantization remains practical for the 7B model but is visibly more harmful for the 1B model on reasoning, math, and code tasks. Ultra-low precision remains challenging: most 2-bit and binary settings collapse to near-random behavior, and W4A4 SmoothQuant produces non-finite perplexity in our evaluation. These results provide an NPU-oriented accuracy map for selecting OpenPangu quantization settings and highlight the persistent difficulty of extreme low-bit compression.

Problem

• OpenPangu
• post-training quantization
• Ascend NPUs
• low-bit compression
• quantization robustness

Innovation

• post-training quantization
• Ascend NPU
• OpenPangu
• low-bit compression
• empirical evaluation