Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
PReCache eliminates redundant trajectory prefilling in multi-LoRA agent systems by sharing a base KV cache while preserving agent-specific representations through low-rank cache precomputation and neutral state reconstruction. This addresses the dual bottlenecks of severe compute redundancy and cross-agent representation contamination during long-horizon collaborative tasks without requiring retraining or architectural modifications. Across multiple agent benchmarks, the framework achieves up to a 3.1× time-to-first-token speedup and a 2.3× throughput gain while holding accuracy degradation to just 1.1 points relative to unshared inference.
Multi-agent LoRA systems can slash time-to-first-token by 3.1× without sacrificing role specialization by decoupling shared base KV caches from precomputed low-rank adapter updates.
Multi-LoRA agent systems enable efficient role specialization by sharing a common backbone model. However, each agent repeatedly processes the growing shared trajectory and constructs its own KV cache, introducing substantial memory and computation redundancy in long-horizon tasks. Existing KV cache sharing methods reduce this repeated prefill, but they either require additional training or architectural constraints or retain substantial model computation. Moreover, direct cache reuse causes the current agent to rely on cache states generated by the previous agent's adapter, weakening the role-specific behavior encoded by its own LoRA. We present PReCache, a training-free KV cache sharing framework with two designs, namely PreLRShared and ReBaseShared, that share the base cache computed using the pretrained weights and precompute a compact agent-specific low-rank (LR) cache. To remove repeated prefill, PreLRShared precomputes each agent's LR cache when the shared context is first processed, allowing the current agent to use its own LR cache without reprocessing context processed by previous agents. To improve sharing accuracy, ReBaseShared reconstructs the shared base cache from adapter-free hidden states, reducing the remaining error caused by the previous agent's adapted representation. To minimize its reconstruction cost, we propose two inference schemes tailored to single-stream inference and concurrent serving, performing the same reconstruction after each agent's turn or alongside its execution, respectively. Across multiple models and agent benchmarks, PreLRShared achieves up to a 3.1x TTFT speedup and a 2.3x improvement in per-request throughput over inference without KV cache sharing. ReBaseShared best preserves accuracy overall among the evaluated cache-sharing methods, with an average drop of only 1.1 points relative to inference without cache sharing.