Search papers, labs, and topics across Lattice.
This paper introduces a prediction-guided runtime system for orchestrating multi-agent workflows on heterogeneous GPU pools, addressing the challenges posed by time-varying loads and resource availability. By estimating device-specific activation latency and model-loading costs, the system constructs an optimized execution graph that enhances scheduling efficiency. The results demonstrate significant improvements, achieving up to 36.8% reduction in end-to-end makespan and 25.9% decrease in p95 completion latency compared to existing schedulers, while also conserving GPU resources.
Optimizing GPU resource allocation can cut multi-agent workflow completion times by nearly 37% while saving substantial GPU usage.
Concurrent multi-agent workflows expose future dependencies and serving-state requirements while running on heterogeneous GPU pools with time-varying load, model residency, and resource availability. The logical workflow defines the required computation, whereas its physical scheduling units, model-lifecycle actions, resource ordering, and placement must be selected according to the observed pool state. We present a prediction-guided runtime that uses workflow forecasts to construct and optimize a physical execution graph. Predictor estimates device-specific activation latency, peak memory, and model-loading cost, then propagates these predictions through workflow dependencies to forecast activation readiness and future model demand. Constructor builds semantics-preserving fusion and model-lifecycle alternatives, while Scheduler jointly optimizes their selection, placement, and execution order based on the live pool state. Across a workload spanning three workflow scenarios on a heterogeneous GPU pool, our system reduces end-to-end makespan and overall p95 completion latency under burst arrivals by up to 36.8% and 25.9%, respectively, over state-of-the-art workflow schedulers. It also saves up to 24.63 GPU-s per completed session.
