Latent Cache Flow: Model-to-Model Communication Without Text

2026-05-19

AI Summary

This work addresses the high latency and information loss inherent in text-based communication among large language model agents, as well as the limited adaptability of existing cache-transfer methods to contextual discrepancies. To overcome these challenges, the authors propose a text-free cross-model communication mechanism that leverages a lightweight adapter (only 13 MB) to directly transmit compressed key-value (KV) cache summaries, enabling efficient state sharing. The core innovations include a joint compression-and-translation framework for KV caches, a context-aware summarization strategy, and a highly compact adapter architecture. Experimental results demonstrate that the proposed method outperforms the 956 MB C2C model in identical-context scenarios and achieves a 23% improvement in communication accuracy alongside an 8.5× speedup in cross-context settings.

Abstract

LLM agents today communicate via text, which incurs considerable latency and information loss due to the need to autoregressively decode the sharer model's state and encode at the receiver model. Recent work such as Cache-to-Cache (C2C; Fu et al., 2026) seeks to exchange KV caches by learning adapters that translate sharer KV matrices to the receiver model. However, the adapters are large and expensive to train, and translate individual tokens, which requires the target context to be identical. This is unsuitable for agent communication, where the LLMs have differing context. We introduce Latent Cache Flow (LCF). To address efficiency, we observe that keys and values can be jointly translated and compressed, reducing the adapter to about 4% of C2C's size. To address differing context, we design the adapter to transmit a summary of new information that the target model does not have. Our early experiments show that a 13 MB LCF adapter can be more accurate than a 956 MB C2C adapter in shared-context settings; for different contexts, LCF is 23% more accurate and 8.5x faster than text-based communication.

Problem

• LLM communication
• KV cache transfer
• context mismatch
• latency
• information loss

Innovation

• Latent Cache Flow
• KV cache compression
• model-to-model communication
• context-aware adaptation
• efficient LLM agents