Search papers, labs, and topics across Lattice.
This paper introduces Spatially Speculative Decoding (SSD), a novel framework that enhances autoregressive image generation by leveraging the 2D spatial locality of visual data. By predicting not only the next token in a 1D sequence but also adjacent tokens in both horizontal and vertical directions, SSD significantly reduces computational bottlenecks during inference. The method achieves up to a 13.3x acceleration in image generation while preserving high fidelity, demonstrating that aligning predictive objectives with spatial geometry can lead to substantial efficiency gains in visual modeling.
Accelerating autoregressive image generation by up to 13.3x, SSD reveals that respecting the 2D structure of images can unlock unprecedented computational efficiencies.
Autoregressive models excel in visual generation by treating images as 1D sequences of discrete tokens, mirroring language modeling. However, this flattening discards the intrinsic 2D spatial locality of visual signals, creating severe computational bottlenecks during inference. We introduce Spatially Speculative Decoding (SSD), a framework that aligns the predictive objective with the natural geometry of images. Rather than predicting only the immediate next token in a 1D sequence, our model simultaneously predicts the adjacent horizontal token and the token directly below it. By capitalizing on this 2D spatial correlation, spatially speculative decoding overcomes the memory wall in visual inference. Our approach accelerates autoregressive image generation by up to 13.3x while maintaining high fidelity on DPG-Bench and GenEval. Our results suggest that respecting the underlying geometry of vision unlocks massive computational efficiencies, paving the way for real-time, high-resolution autoregressive generative models.
