Search papers, labs, and topics across Lattice.
The paper introduces Position-Aware Drafting for generative Recommendation (PAD-Rec), a method to accelerate LLM-based generative list-wise recommendation using speculative decoding. PAD-Rec enhances the draft model with item position embeddings to encode within-item token slots and step position embeddings to account for depth-dependent uncertainty. Experiments on real-world datasets demonstrate up to 3.1x wall-clock speedup and a 5% average gain over strong speculative decoding baselines, while maintaining recommendation quality.
Achieve up to 3.1x faster LLM-based recommendations by making speculative decoding aware of token positions within items and speculation depth.
Large language model (LLM)-based generative list-wise recommendation has advanced rapidly, but decoding remains sequential and thus latency-prone. To accelerate inference without changing the target distribution, speculative decoding (SD) uses a small draft model to propose several next tokens at once and a target LLM to verify and accept the longest prefix, skipping multiple steps per round. In generative recommendation, however, each item is represented by multiple semantic-ID tokens, often with separators, and current drafts typically treat these tokens uniformly. This overlooks two practical facts: (i) a token's semantics depend on its within-item slot, and (ii) uncertainty tends to increase with speculation depth. Without modeling these effects, SD's speedups can be limited. We introduce PAD-Rec, Position-Aware Drafting for generative Recommendation, a lightweight module that augments the draft model with two complementary signals. Item position embeddings explicitly encode the within-item slot of each token, strengthening structural awareness. Step position embeddings encode the draft step, allowing the model to adapt to depth-dependent uncertainty and improve proposal quality. To harmonize these signals with base features, we add simple gates: a learnable coefficient for item slots and a context-driven gate for draft steps. The module is trainable, easy to integrate with standard draft models, and adds negligible inference overhead. Extensive experiments on four real-world datasets show up to 3.1x wall-clock speedup and about 5% average wall-clock speedup gain over strong SD baselines, while largely preserving recommendation quality.
