Squeezing the Cache, Preserving the Truth: Monotonic Equipotential Allocation with Geodesia-KV

2026-08-19

AI Summary

本文提出Geodesia-KV，通过单调节块精度分配等方法解决KV缓存压缩性能评估问题，显著优于其他方法。

Abstract

Current assessment of KV-cache compression performance confuses resident bits with read bandwidth and is affected by the artifacts of chunked teacher-forcing. We present Geodesia-KV, a family of training-free KV cache policies based on monotonic block-wise precision allocation, exact rate-distortion residuals, and query-sparse reading, enabling proper hardware-ready compression. With proper separation of resident and read bits and causal evaluation, we show that Geodesia-KV significantly outperforms other approaches. Specifically, on WikiText-2 with 16k context, the 5-bit operating point of Geodesia-KV results in lower perplexity at lower bitrate than KIVI-4 on Qwen. In addition, our compressed-Quest version delivers improved perplexity and reduces resident (9.83 vs 16.25 bits/value) and read rates (1.95 vs 2.32 bits/value) over baseline sparse methods on PG-19. As Geodesia-KV is implemented as native GeodesiaKVCacheManager plug-in of vLLM, Geodesia-KV fully removes the need for dense cache residency via monotonic bit demotion. With the full consumer hardware evaluation, Geodesia-KV leads to 1M-token context generation on a single 16 GiB GPU with up to 71.7% peak VRAM savings on all leading architectures (Qwen, Llama, DeepSeek).

Problem

• KV-cache compression
• resident bits
• read bandwidth
• hardware-ready compression
• performance evaluation

Innovation

• monotonic block-wise precision allocation
• exact rate-distortion residuals
• query-sparse reading