Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving

2026-08-09

AI Summary

This work addresses the efficiency and storage challenges of weight quantization and expert loading in large-scale Mixture-of-Experts models by proposing an end-to-end quantized inference framework. The key innovation lies in constraining the PTQTP method to a nine-level balanced ternary quantizer with a fixed ratio of 3, which enables lossless folding of ternary planes into a unified byte format of 4.0625 bits per weight. This format facilitates seamless co-design across disk storage, caching, and SIMD compute kernels. Integrated with SSD streaming loading and MXFP4 conversion, the approach achieves a 9% reduction in model file size and a 6.7% speedup in decoding on DeepSeek-V4-Flash-0731, while attaining an MMLU subset score of 86 (baseline: 84) and exact match with the official API on initial steps across five benchmark tasks.

Abstract

PTQTP decomposes LLM weight matrices into two ternary (trit) planes with two free per-group scales. Tying the scales to a fixed ratio of three collapses the decomposition into a single uniform nine-level quantizer, a known balanced-ternary identity. To our knowledge, at the time of writing, this work is the first to impose that identity as a constraint inside PTQTP's solver. The two trit planes then fold losslessly into one 4-bit code plane that we make the persistent serving representation: disk bytes, expert-cache bytes, and kernel input are the same 4.0625-bits/weight blocks, consumed in one integer dot pass. For this conjunction (ratio-3 nine-level code, CPU-SIMD kernels, SSD expert streaming, identical persistent bytes) we likewise found no precedent. We apply this to the routed experts of DeepSeek-V4-Flash-0731, a 284B-A13B mixture-of-experts model, quantizing in one shot from the released MXFP4 expert weights and streaming experts from SSD on a 64 GB laptop. Against a 4.5-bit Q4_K baseline, measured one process per fixture with an expert-lossless anchor arm as reference control, the tied model matches the official serving API on 5/5 fixtures at step 0 (Q4_K: 4/5) and 12/14 captured continuation steps (11/14), scores 86 vs. 84 on a 100-item MMLU subset, decodes 6.7% faster in decode phase, and ships 9% smaller files: no detected fidelity difference at these small evaluation sizes, and every fixture-level difference between the arms traces to a single measured near-tie cell. The tied fit nevertheless shows higher weight-reconstruction error and worse perplexity, a measured dissociation between proxy metrics and reference fidelity. A cumulative trunk-ternarization ladder and bitwise-pinned aarch64/x86-64 kernels complete the report. All code, formats, and evaluation artifacts are open source in the fucina inference stack.

Problem

• quantization
• mixture-of-experts
• disk-streaming
• uniform quantizer
• low-bit inference

Innovation

• Tied Trit-Planes
• uniform nine-level quantizer
• disk-streamed MoE
• lossless folding
• PTQTP