Search papers, labs, and topics across Lattice.
This paper introduces TIGER, a novel framework for multimodal speculative decoding that enhances the efficiency of visual language models (VLMs) by dynamically selecting context-relevant visual tokens based on the drafter's textual state. By employing acceptance-aligned group-based policy training, TIGER optimizes the drafter to produce speculative continuations that are more likely to be verified by the target model, thus maximizing the accepted prefix length. Experimental results demonstrate that TIGER significantly improves both the accepted prefix length and speculative speedup, while maintaining competitive accuracy in downstream tasks.
TIGER achieves remarkable speedup in multimodal generation by intelligently routing visual tokens based on textual context, outpacing traditional methods.
Speculative decoding accelerates autoregressive generation by letting a lightweight drafter propose multiple tokens that are verified by a larger target model. Although effective for text-only LLMs, speculative decoding yields limited gains in VLMs because drafters often diverge on vision-critical content, while existing multimodal acceleration methods do not directly address irrelevant visual evidence or optimize the verifier-accepted prefix length that governs speedup. We propose TIGER, a Text-conditioned vIsual GatEd Routing framework for multimodal speculative decoding. TIGER dynamically selects a sparse set of context-relevant visual tokens based on the drafter's current textual state, rather than expose the full visual token set or a fixed compressed interface. To better align training with inference-time efficiency, we optimize the drafter with acceptance-aligned group-based policy training using verifier-derived rewards based on accepted prefix length, built on top of distillation warm start with KL anchoring. This encourages the drafter not only to imitate the target model, but also to produce speculative continuations that survive verification for longer prefixes. Experiments show that TIGER yields consistent gains in accepted prefix length and speculative speedup under exact verifier-side speculative decoding, while achieving favorable quality-latency trade-offs with comparable downstream accuracy in visual-routing analyses.
NUS