Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces PASCAL, a phase-aware scheduling algorithm designed to optimize the serving of reasoning-based LLMs by explicitly differentiating and prioritizing the reasoning phase to minimize Time-To-First-Token (TTFT). PASCAL employs a hierarchical scheduler with instance-level placement, intra-instance execution management, and dynamic migration at phase boundaries to balance load and reduce interference. Experiments using DeepSeek-R1-Distill-Qwen-32B show that PASCAL reduces tail TTFT by up to 72% while preserving answering phase SLO attainment, highlighting the benefits of phase-aware scheduling.
Reasoning-heavy LLMs can cut their "thinking" time by up to 72% with a new scheduling algorithm that understands when the model is reasoning vs. answering.
The emergence of reasoning-based LLMs leveraging Chain-of-Thought (CoT) inference introduces new serving challenges, as their extended reasoning phases delay user-visible output and inflate Time-To-First-Token (TTFT). Existing LLM serving frameworks fail to distinguish between reasoning and answering phases, leading to performance degradation under GPU memory constraints. We present PASCAL, a phase-aware scheduling algorithm that prioritizes reasoning to reduce TTFT while using controlled preemption and token pacing during answering to preserve Quality-of-Experience (QoE). Our hierarchical scheduler combines instance-level placement with intra-instance execution and enables dynamic migration at phase boundaries to balance load and reduce interference. Across benchmarks using DeepSeek-R1-Distill-Qwen-32B, PASCAL reduces tail TTFT by up to 72% while maintaining answering phase SLO attainment, demonstrating the importance of phase-aware scheduling for reasoning-based LLM deployment.