Search papers, labs, and topics across Lattice.
This paper introduces Parent-Conditioned Drafting Tree (PCTree), an innovative approach that enhances the speculative decoding process in large language models (LLMs) by allowing multiple parent-consistent continuations without the need for retraining. By leveraging the conditional structure learned by existing semi-autoregressive drafters like DSpark, PCTree transforms the linear drafting process into a branching tree structure, enabling more efficient verification of alternative continuations. The method achieves significant speedup gains over traditional autoregressive decoding, with improvements ranging from 3.1% to 29.5% across various benchmarks, demonstrating the potential for enhanced inference efficiency in LLMs.
Parent-Conditioned Drafting can boost LLM inference speed by up to 29.5% while increasing the effective acceptance length of generated outputs.
Speculative decoding accelerates LLM inference only when drafted continuations survive target-model verification. Semi-autoregressive drafters such as DSpark predict an entire token block with one backbone forward and refine it with a lightweight Markov head. However, DSpark decodes this block as a single chain, so an early mismatch invalidates the remaining suffix and limits the benefit of large draft blocks. We show that the conditional structure already learned by DSpark can support multiple parent-consistent continuations without retraining or additional backbone passes. We introduce Parent-Conditioned Drafting Tree (PCTree), which uses the pretrained Markov head to score alternative children separately for each concrete parent and allocates a fixed verification budget to the most probable paths. This converts DSpark's linear draft into a tree while preserving its one-pass parallel backbone. Across Qwen3-{4B,8B,14B} and nine benchmarks, at $B{=}7$, measured speedup gains over autoregressive (AR) decoding, relative to matched DSpark, range from $3.1\%$ to $29.5\%$. On Qwen3-4B GSM8K at $B{=}16$, PCTree increases mean acceptance length from $9.41$ to $11.16$ and three-run mean AR speedup from $6.14{\times}$ to $6.60{\times}$. These show that parent-conditioned branching can turn conditional capacity already present in a semi-autoregressive drafter into end-to-end inference gains through an inference-only change.
