Search papers, labs, and topics across Lattice.
This paper introduces Simthesizer, an agent-driven simulation framework designed to address the rapid evolution of LLM serving systems, which traditional simulators struggle to keep pace with. By employing a composable simulator infrastructure that represents the entire serving workflow as a dynamic graph, Simthesizer enables the efficient integration of new features without the need for extensive rewrites. The framework demonstrates significant performance improvements, achieving up to 284.96x faster simulation speeds and a lower average throughput error of 2.51% compared to existing simulators.
Simthesizer achieves up to 284.96x faster simulation speeds while maintaining a mere 2.51% throughput error, revolutionizing how we model LLM serving systems.
System-level simulation is an essential tool for exploring the rapidly expanding design space of LLM serving systems, where real deployments remain costly and often infeasible. However, modern LLM serving now evolves faster than human-driven simulator development can track, and emerging workloads and mechanisms, from agentic workflows to disaggregated serving, no longer fit the monolithic simulation pipeline that existing simulators assume. Each new mechanism therefore demands an invasive rewrite, leaving a widening development gap between deployed serving systems and the simulators that model them. To close this gap, we present Simthesizer, a framework that realizes agent-driven simulator development. Simthesizer introduces a composable simulator infrastructure that uniformly expresses the complete serving workflow, including the control decisions that coordinate it, and realizes it as a unified dynamic graph in Simthesizer simulator. Synthesizer agent, a harnessed coding agent, then lowers natural-language feature requests onto this abstraction under simulator-specific guardrails and fidelity validation, evolving one shared simulator instead of building a new one for every feature. Under the same coding agent and harnesses, extensions built on Simthesizer follow a vLLM-based real system with 2.51% average throughput error, versus 6.03% for extensions built on existing simulators. On identical workloads, Simthesizer also simulates up to 284.96x and 23.19x faster than two state-of-the-art simulators, LLMServingSim2.0 and Vidur, respectively.
