Search papers, labs, and topics across Lattice.
The paper introduces Hive, a multi-agent infrastructure designed to enable algorithm- and task-level scaling in LLM-based agentic systems. Hive addresses cross-path redundancy in algorithm-level scaling with a Logits Cache, and optimizes resource allocation in task-level scaling with Agent-Aware Scheduling. Experiments demonstrate that Hive's Logits Cache achieves up to 1.76x speedup for re-sampling, while Agent-Aware Scheduling reduces hotspot miss rates by up to 51%.
LLM agent systems can achieve up to 76% speedups and significantly reduced hotspot miss rates by intelligently caching logits and scheduling compute resources based on agent behavior.
Large language models are increasingly deployed as complex agentic systems that scale with task complexity. While prior work has extensively explored model- and system-level scaling, algorithm- and task-level scaling remain largely unaddressed, constraining the full potential of agentic systems. At the algorithm level, allocating additional inference-time computation can enhance workflow capacity but introduces cross-path redundancy: overlapping computations across multiple reasoning branches. At the task level, complex tasks can be decomposed into subproblems and delegated across multiple agents for improved scalability and parallelism. However, existing infrastructures' scheduling is unaware of the existence of multiple agents, missing opportunities to optimize resource allocation. We propose Hive, a multi-agent infrastructure that enables algorithm- and task-level scaling. Hive features a description frontend that captures per-agent behavior and supports test-time scaling algorithms. Leveraging this specification, our backend introduces two key mechanisms: Logits Cache that reuses intermediate logits across redundant sampling paths to mitigate cross-path redundancy at the algorithm level, and Agent-Aware Scheduling that efficiently allocates compute and KV-cache resources according to agent contributions at the task level. Experiments show that Logits Cache achieves an average speedup of $1.11\times$-$1.76\times$ for re-sampling, and Agent-Aware Scheduling reduces the hotspot miss rate by $33\%$-$51\%$.
