AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization

2026-05-09

AI Summary

Existing 4-bit post-training quantization methods struggle to simultaneously achieve high accuracy and compression efficiency under extremely low computational overhead. This work proposes AAAC, the first approach to incorporate an activation-aware adaptive codebook mechanism into 4-bit quantization. Specifically, AAAC introduces two lightweight, learnable scalar codebooks per layer and dynamically selects the optimal one based on activation-weighted reconstruction error. Crucially, it repurposes the otherwise unused sign bit to encode the codebook selection, thereby incurring zero additional storage cost. Requiring only 3–30 minutes for quantization, AAAC substantially outperforms state-of-the-art baselines—including AWQ, GPTQ, and OmniQuant—across multiple mainstream large language models, achieving higher accuracy at orders of magnitude lower computational cost.

Abstract

Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference. Existing PTQ methods, such as AWQ and GPTQ, improve how weights are mapped onto a fixed 4-bit grid through scaling, clipping, or error compensation. To further improve accuracy, methods such as OmniQuant and QuIP\# uses gradient-assisted algorithms at the cost of hours of quantization time. In this work, we propose AAAC (Activation-Aware Adaptive Codebooks), a lightweight method for 4-bit LLM weight quantization. AAAC replaces the fixed scalar codebook used in standard quantization with two small learned scalar codebooks (64 bytes) per layer. Each group of weights selects the codebook that minimizes activation-weighted reconstruction error, encoding the choice in the unused sign bit of the group's positive scale and adding zero storage overhead. AAAC completes in 3--30 minutes on a single GPU, and adds no memory beyond the model itself. We evaluate against AWQ, GPTQ, IF4, GPTVQ, OmniQuant, SqueezeLLM, and QuIP\# across model families. AAAC outperforms baselines at orders-of-magnitude less quantization time.

Problem

• LLM quantization
• 4-bit weight quantization
• post-training quantization
• codebook adaptation
• activation-aware quantization

Innovation

• Activation-Aware
• Adaptive Codebooks
• 4-bit Quantization
• Post-Training Quantization
• Zero Storage Overhead