Search papers, labs, and topics across Lattice.
This paper introduces Festina, a profiling-guided control plane designed to optimize energy consumption in serverless LLM serving on shared GPUs. By coordinating request placement, SM partitioning, and GPU operating points, Festina achieves significant energy savings without compromising service level objectives (SLOs). The system demonstrates a reduction in energy consumption by up to 56% compared to state-of-the-art LLM serving systems while maintaining SLO attainment within a 2% margin.
Festina slashes energy consumption by up to 56% in serverless LLM serving while ensuring SLOs are met, revolutionizing energy optimization in cloud workloads.
As LLM inference becomes a major cloud workload, its growing energy footprint makes cluster-wide energy optimization increasingly important. Serverless LLM serving helps platforms absorb traffic volatility by elastically sharing GPU resources across models, but this sharing also makes energy optimization difficult. Multiple co-resident models run under one device-wide operating point, while their resource demands and latency slack change across execution phases and load conditions. As a result, minimizing energy requires coordinated scheduling across request placement, runtime resource adaptation, and workload consolidation. We present Festina, a profiling-guided, power-aware control plane to minimize cluster-wide energy for serverless LLM serving. Unlike common global-local schedulers that focus on throughput or tail latency, Festina makes energy-first decisions by jointly coordinating request placement, SM partitioning, and GPU operating points under TTFT/TBT SLOs. In our system, a lightweight global scheduler performs fast, SLO-safe, energy-aware placement using constant-time lookups from offline profiles and GPU state summaries. On each GPU, a phase-aware local scheduler continuously adapts task batching and compute resources to minimize power consumption. Festina further performs energy-aware workload consolidation to reduce GPUs'static power consumption via SLO-aware migration. Comparison with four SOTA LLM serving systems and one DVFS-augmented system demonstrates that Festina reduces energy consumption by up to 56% while maintaining parity in SLO attainment (within a 2% margin)
