Addition is almost all you need: Compressing neural networks with double binary factorization

2025-05-16

AI Summary

To address the severe accuracy degradation and inefficiency trade-off in binary quantization of large language models (LLMs), this paper proposes Dual Binary Factorization (DBF). DBF decomposes each weight matrix into the product of two learnable binary matrices and a pair of channel-wise scaling vectors, replacing multiplications with additions for highly efficient inference. It introduces the first dual-binary factorization architecture, enabling layer-wise adaptivity and fine-grained, lossless control over compression ratios. DBF breaks the accuracy ceiling of conventional single-binary quantization: at 1 bit/weight, it significantly outperforms state-of-the-art binary methods; at 2 bits/weight, it matches or exceeds the performance of advanced quantization schemes such as QuIP# and QTIP. Crucially, DBF achieves ~32× storage compression while incurring only additive computation—unifying high accuracy, ultra-low computational cost, and extreme model compression.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Binary quantization approaches, which replace weight matrices with binary matrices and substitute costly multiplications with cheaper additions, offer a computationally efficient approach to address the increasing computational and storage requirements of Large Language Models (LLMs). However, the severe quantization constraint ($pm1$) can lead to significant accuracy degradation. In this paper, we propose Double Binary Factorization (DBF), a novel method that factorizes dense weight matrices into products of two binary (sign) matrices, each accompanied by scaling vectors. DBF preserves the efficiency advantages of binary representations while achieving compression rates that are competitive with or superior to state-of-the-art methods. Specifically, in a 1-bit per weight range, DBF is better than existing binarization approaches. In a 2-bit per weight range, DBF is competitive with the best quantization methods like QuIP# and QTIP. Unlike most existing compression techniques, which offer limited compression level choices, DBF allows fine-grained control over compression ratios by adjusting the factorization's intermediate dimension. Based on this advantage, we further introduce an algorithm for estimating non-uniform layer-wise compression ratios for DBF, based on previously developed channel pruning criteria. Code available at: https://github.com/usamec/double_binary

Problem

• Reducing computational and storage needs in LLMs efficiently
• Minimizing accuracy loss from binary quantization constraints
• Enabling flexible compression ratio control in neural networks

Innovation

• Double Binary Factorization for weight matrices
• Scaling vectors enhance binary quantization efficiency
• Fine-grained compression ratio control via factorization