Search papers, labs, and topics across Lattice.
This paper introduces receiver-conditioned latent communication, where agents filter their KV cache transmissions based on the receiver's stated information needs, significantly reducing communication overhead. By leveraging sender attention weights to identify relevant KV cache entries, CacheBack achieves a 94% reduction in transmitted state on FanOutQA, leading to a 14.7 percentage point accuracy improvement and a 3.2x latency reduction compared to text-based communication. This approach demonstrates broad applicability across diverse transformer architectures and attention mechanisms.
Latent communication can be compressed by 75% without accuracy loss by conditioning KV cache transmissions on receiver information needs.
Multi-agent systems distribute large contexts across agents that communicate to solve a task. Text messages are compact but require decoding and may omit evidence the receiving agent needs. Recent latent communication instead transfers KV caches. This avoids text generation and can improve accuracy and latency. However, a full KV cache grows linearly with both the context an individual agent processes, and the number of agents that coordinate together. This raises memory and context costs, often far exceeding available GPU resources and context window sizes. Our key observation is that agents need only send what the receiving agent requires for its local task -- which we call receiver-conditioned communication. The receiver agent passes the sender a small description of its information needs, which serves to filter and compress the sender agent's KV cache. CacheBack is a simple, robust, training-free instance of receiver conditioning based on the sender's attention weights. On FanOutQA, CacheBack with Qwen 3 removes 75% of the state the agent would otherwise receive, improving accuracy by 14.7 percentage points and reducing median task-completion latency by 3.2x relative to text communication. We show comparable improvements across model families that span dense Transformers, Mamba-attention hybrids, and sliding-window attention.
Amazon Science