Search papers, labs, and topics across Lattice.
This paper investigates inference-time parallelism in multi-agent systems driven by large language models (LLMs), introducing a framework called TIPEX that unifies two levels of parallelism: Replica Parallelism and Structural Parallelism. By systematically analyzing their interrelationships and coordination, the authors demonstrate that TIPEX can significantly enhance accuracy and reduce latency during inference, albeit with increased token consumption. Experimental results on the GAIA benchmark reveal that intermediate complexity tasks benefit the most from coordinated parallel strategies, while excessive parallelism can lead to diminishing returns.
Coordinated parallelism in multi-agent LLM systems can boost accuracy and cut latency, but only if applied judiciously鈥攐verdoing it may backfire.
Large language model (LLM)-driven multi-agent systems typically require multiple model invocations and complex coordination during inference, and their execution strategies directly affect system accuracy, latency, and computational cost. Parallel execution provides a means to improve inference-time efficiency. From the perspective of inference-time execution, this paper models parallelism in multi-agent systems as two distinct levels of decision processes: Replica Parallelism, which explores multiple complete solution paths at the task level, and Structural Parallelism, which enables concurrent execution within a single solution path through task decomposition. However, the roles of different forms of parallelism and their interrelationships still lack systematic study in terms of unified organization and coordination. We therefore propose TIPEX, a controllable execution framework that unifies these two levels of parallelism and coordinates their roles within the inference process under a unified execution semantics while supporting systematic combinations and analyses of different parallel strategies and parameter configurations. Systematic experiments on the GAIA benchmark demonstrate that inference-time parallelism can significantly improve accuracy and reduce end-to-end latency at the cost of increased token consumption. Further analysis shows that Replica and Structural Parallelism exhibit complementary effects across task complexities, with tasks of intermediate difficulty benefiting most from their coordination, while overly aggressive parallel strategies do not necessarily yield better performance.
