Search papers, labs, and topics across Lattice.
AAFLOW+ introduces a stateful operator abstraction that transforms key-value (KV) cache management into a distributed systems object, enabling efficient multi-agent workflows. By constructing communication-aware graphs, AAFLOW+ optimizes the reuse of model state and context, significantly reducing the need for costly recomputation. The approach achieves remarkable performance improvements, including a reduction in time-to-first-token (TTFT) by up to 50.2x and multi-agent compute costs by up to 7.63x at 16-agent scale, demonstrating the superiority of KV transmission over traditional text passing in high-bandwidth environments.
AAFLOW+ slashes multi-agent compute costs by over 7x while enabling zero-copy context sharing, revolutionizing how agents manage shared state.
Multi-agent LLM systems increasingly integrate retrieval, planning, and reasoning, but remain fundamentally text-centric, requiring agents to repeatedly recompute shared context through expensive prefill. Although single-request inference is known to be accelerated by KV-cache management, it is usually restricted to local serving scopes. We introduce AAFLOW+, a stateful extension of agentic workflow operators that makes KV cache a first-class distributed systems object. AAFLOW+ builds processes into communication-aware graphs that concurrently optimize data, prompts, and reusable model state. It also provides operators for KV materialization, transfer, fork, composition, and eviction. Its runtime enables zero-copy, transfer-aware execution, allowing agents to reuse long context without recomputation. AAFLOW+ reduces TTFT by up to 50.2x, achieves up to 7.63x reduced multi-agent compute cost at 16-agent scale, reduces KV memory by 1.72-6.10x, and increases throughput by more than 7.74x, based on an analytical cost model parameterized by empirical hardware microbenchmarks. The results demonstrate that KV transmission outperforms recomputation on networks with moderate to high bandwidth, making sure KV-state sharing greatly increases efficiency in multi-agent LLM systems by replacing text passing.
