Search papers, labs, and topics across Lattice.
3
0
5
11
LLM decoder states exhibit distinct informational and geometric compartments, enabling principled layer selection for speculative decoding that surpasses heuristic approaches.
LLMs can predict multiple tokens in parallel without any training, simply by cleverly probing their embedding space with dynamically generated mask tokens.
By enabling draft models to "contemplate the future," ConFu achieves significant speedups in speculative decoding, outperforming EAGLE-3 by 8-11% on Llama-3 models.
