Search papers, labs, and topics across Lattice.
This paper introduces Xema, a novel memory-efficient diffusion serving system designed to optimize GPU memory usage during high-resolution image and long video generation. By leveraging predictable tensor lifetimes and implementing trace-guided memory optimization, Xema minimizes memory pressure and fragmentation, allowing for more efficient resource allocation. The system demonstrates significant performance improvements, achieving up to 3.7x better service level objective (SLO) attainment while drastically reducing planning time from 6.3 hours to just 197 seconds compared to traditional methods.
Xema achieves a staggering 3.7x improvement in service level objectives for diffusion models while slashing planning time from hours to mere minutes.
Diffusion models are increasingly deployed as production visual-generation services, where serving high-resolution image and long video generation is often limited by GPU memory. Popular memory-saving techniques such as weight offloading, sharding, and VAE slicing are often not practical because they tend to introduce significant performance overhead. In this paper, we present Xema, a memory-efficient diffusion serving system that exploits predictable tensor lifetimes for trace-guided memory optimization. For each request template, Xema derives an offline memory trace to identify short memory-pressure intervals and applies memory mitigation only within these intervals and only by the amount needed to fit the target GPU budget. Xema further constructs a static memory layout for tensors with predictable lifetimes, reducing fragmentation-induced reserved memory and making offline memory reasoning reliable at runtime. Built on this memory optimization layer, Xema introduces an offline planner that jointly selects parallelism, concurrency, and memory control under GPU memory and SLO constraints. The selected plan is stored in a plan table and directly used by the online serving runtime. We implement Xema on production diffusion pipelines and evaluate it with Flux.2, CogVideoX-5B, and LTX-2. Compared with existing serving configurations, Xema improves SLO attainment by up to 3.7x and reduces planning cost from 6.3 hours to 197 seconds compared with grid search.
