CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agents

2026-08-07

AI Summary

This work addresses the persistent growth of key-value (KV) cache in ReAct (Reasoning and Acting) agents during multi-turn interactions, which leads to substantial memory and computational overhead due to accumulated trajectory data. The authors propose the first lifecycle-aware mechanism triggered by tool-call “commit” events, which distinguishes between dormant and safely removable KV pages by comparing model states before and after commit actions. Integrating paged memory management, a greedy joint eviction test, and a protected caching strategy, the method dynamically prunes redundant KV entries while preserving absolute positional index consistency. Evaluated across multiple benchmarks, this approach significantly reduces memory consumption, accelerates end-to-end inference, and achieves higher task accuracy than existing methods even under aggressive compression ratios.

Abstract

Multi-turn Reasoning-and-Acting (ReAct) agents accumulate growing trajectories of reasoning, tool calls, and observations. Their key-value (KV) caches grow accordingly, increasing memory use and attention cost during model inference. Existing KV cache compression methods reduce these costs by evicting states with low attention scores. However, low attention in the current turn does not imply future irrelevance, as temporarily inactive information may become important later. Snapshot-based eviction methods therefore do not explicitly distinguish temporarily dormant information from information that appears to have completed its role. In this paper, we present CommitKV, which identifies KV lifecycles through commit transitions. Specifically, CommitKV first divides completed agent events into token pages and compares each eligible page's deletion effect before a tool-call commit and after the commit's returned observation has been incorporated. Based on these paired measurements, CommitKV distinguishes dormant pages from high-to-low completion candidates. It then applies a greedy joint test, accepting candidates for retirement only when their combined post-commit effect remains bounded. Finally, at a later compression checkpoint, accepted pages are excluded, a bounded set of pages awaiting post-commit measurement is protected, and the remaining KV states are retained within the cache budget using the same token indices for keys, values, and absolute positions. These mechanisms ensure that CommitKV can distinguish dormant information from information that has completed its observed role and can be safely removed. Experiments on various benchmarks show that CommitKV reduces agent memory use, accelerates end-to-end inference, and achieves higher accuracy than existing KV cache compression methods.

Problem

• KV cache compression
• multi-turn agents
• lifecycle awareness
• commit transitions
• memory efficiency

Innovation

• KV cache compression
• commit transitions
• multi-turn agents
• lifecycle-aware eviction
• ReAct agents