Search papers, labs, and topics across Lattice.
PromptTuner is introduced as an SLO-aware elastic system designed to optimize LLM prompt tuning by minimizing resource provisioning costs while meeting user-defined service level objectives. The system incorporates a Prompt Bank to accelerate convergence by identifying efficient initial prompts and a Workload Scheduler for rapid resource allocation. Evaluations demonstrate that PromptTuner significantly reduces SLO violations (4.0x-7.9x) and lowers costs (1.6x-4.5x) compared to INFless and ElasticFlow.
PromptTuner slashes SLO violations by up to 7.9x and costs by 4.5x in LLM prompt tuning, outperforming existing resource management systems.
Prompt tuning has become a prominent strategy for enhancing the performance of Large Language Models (LLMs) on downstream tasks. Many IT enterprises now offer Prompt-Tuning-as-a-Service to fulfill the growing demand for prompt tuning LLMs on downstream tasks. Their primary objective is to satisfy users Service Level Objectives (SLOs) while reducing resource provisioning costs. Nevertheless, our characterization analysis for existing deep learning resource management systems reveals that they are insufficient to optimize these objectives for LLM prompt tuning workloads. In this paper, we introduce PromptTuner, an SLO-aware elastic system to optimize LLM prompt tuning. It contains two innovations. (1) We design a Prompt Bank to identify efficient initial prompts to expedite the convergence of prompt tuning. (2) We develop aWorkload Scheduler to enable fast resource allocation to reduce the SLO violation and resource costs. In our evaluation, PromptTuner reduces SLO violations by 4.0x and 7.9x, and lowers costs by 1.6x and 4.5x, compared to INFless and ElasticFlow respectively.
