Search papers, labs, and topics across Lattice.
This paper introduces ServerlessT2I, a novel serverless-native system that optimizes text-to-image (T2I) workflows by decomposing them into loosely coupled model functions for independent management and scheduling. By leveraging idle GPU memory and implementing a fair scheduling mechanism, ServerlessT2I achieves significant efficiency gains, sustaining up to 2脳 higher request rates and reducing GPU resource usage by up to 3脳 while meeting service level objectives. These advancements address the limitations of monolithic GPU function deployments, enhancing scalability and fairness in multi-tenant environments.
ServerlessT2I can double the request rates of text-to-image workflows while slashing GPU resource consumption by up to three times.
Text-to-image (T2I) workflows are increasingly deployed on serverless platforms because users often compose customized workflows and invoke them intermittently. Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together. This monolithic design obscures workflow structure, inflates scaling overhead, forces users to manage low-level GPU coordination, and limits fine-grained fairness in multi-tenant clusters. In this paper, we present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled. By explicitly managing individual model execution, ServerlessT2I enables per-model scaling, declarative workflow composition, transparent GPU-resident communication, and fairness-aware scheduling. To make this decomposition efficient, ServerlessT2I harvests slack GPU memory left idle by compute-bound T2I inference to build a data plane that reduces model loading and data communication overheads. \sys{} further introduces a fair scheduler for multi-tenant serving. Using production traces, ServerlessT2I sustains up to 2$\times$ higher request rates than existing T2I workflow serving systems with the same GPU budget; for a fixed request rate, it saves up to 3$\times$ GPU resources while satisfying service level objectives (SLOs).
