Search papers, labs, and topics across Lattice.
Optimus is introduced, a serving system for diffusion LLMs that elastically adapts decoding granularity to runtime load by balancing GPU utilization and token efficiency. It combines chunked decoding for fine-grained execution with saturation-aware scheduling, a closed-loop mechanism that selects chunk sizes based on runtime conditions. Experiments demonstrate up to 6.1x throughput improvement over AR decoding and 4.3x over fixed-block diffusion LLMs, while maintaining stable performance and improving serving capacity under latency constraints.
Diffusion LLMs can achieve up to 6.1x higher throughput than autoregressive models by dynamically adjusting decoding granularity based on real-time load, a feat unattainable with fixed-block approaches.
Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential execution, while diffusion LLMs (DLLMs) improve throughput by decoding multiple tokens per iteration. However, fixed block-size diffusion decoding exhibits strong load sensitivity: large blocks exploit idle GPU resources under low load, but saturate early and incur substantial redundant computation under high load. As a result, throughput gains vanish beyond saturation, and no single decoding granularity performs well across dynamic serving workloads. We present Optimus, a serving system that enables elastic decoding for diffusion LLMs by dynamically adapting decoding granularity to runtime load. The key idea is to treat decoding granularity as a runtime control variable, balancing GPU utilization and token efficiency. Optimus combines chunked decoding, which enables fine-grained execution without retraining, with saturation-aware scheduling, a closed-loop mechanism that selects chunk sizes based on runtime conditions. Together with system-level optimizations and customized attention kernels, Optimus achieves significant performance improvements while preserving model accuracy. Experiments show that Optimus delivers up to 6.1x throughput improvement over AR decoding and 4.3x improvement over fixed-block diffusion LLM, while maintaining stable performance across diverse load regimes and improving end-to-end serving capacity under latency constraints. The source code is available at https://github.com/dubcyfor3/Optimus.
NUS