Cache Merging as a Convergent Replicated State for Multi-Agent Latent Reasoning

2026-07-01

AI Summary

This work addresses the non-determinism in multi-agent implicit reasoning caused by order-sensitive KV cache merging. The authors propose CanonicalMerge, a method that models cache merging as a set-based CvRDT (convergent replicated data type) state structure. By sorting intermediate-layer key vectors according to their ℓ² norms and employing a content-addressable mechanism, CanonicalMerge achieves input-order-independent merging that satisfies commutativity and guarantees byte-level equivalence. The approach matches the performance of the optimal BagMerge across diverse model scales and inference budgets, significantly outperforms training-agnostic output fusion baselines on HotpotQA (+45 points), and inherently supports automatic absorption of duplicate caches.

Abstract

Multi-agent latent reasoning composes agents' KV-caches into one context for a final agent. Prior work (Agent Primitives) does this by concatenating caches along the sequence axis with RoPE re-encoding, which we call BagMerge. BagMerge is non-commutative, and the best input ordering is unpredictable, shifting with the regime, the latent-step budget, and the model scale. We make this exchange a convergent replicated state. First, CanonicalMerge fixes the layout by content: ordering caches by mean K-norm at a middle layer renders the merged cache byte-identical under any input permutation, verified algorithmically (arity N<=5) and bit-for-bit on real Qwen3-1.7B and 4B state. Second, we separate the replicated state from decode-time layout: the state is a set of content-addressed latent fragments whose merge is set union, a state-based CvRDT (commutative, associative, idempotent, absorbing), and CanonicalMerge is its deterministic render. Because the render is byte-equivalent, every N=2 accuracy number carries over unchanged and re-delivered duplicates are absorbed rather than re-concatenated. On a partitioned-reasoning benchmark, CanonicalMerge matches the best BagMerge ordering in every regime-by-budget-by-ordering cell without knowing which order is best, trading a small, statistically insignificant accuracy margin for an unconditional structural guarantee. The behaviour transfers to real multi-document QA (HotpotQA), while the closest training-free output-fusion baseline (PackLLM) loses by 45 points at matched budget, placing cache-level merging in a regime distinct from output-level fusion. Finally, at k>2 the approach transports and colocates latent traces but does not by itself compose them, which we characterize to motivate future work.

Problem

• multi-agent latent reasoning
• KV-cache merging
• convergent replicated state
• commutativity
• cache composition

Innovation

• CanonicalMerge
• convergent replicated state
• CvRDT
• KV-cache merging
• latent reasoning