Search papers, labs, and topics across Lattice.
This paper introduces SJD-PAC, an improved Speculative Jacobi Decoding framework for accelerating text-to-image synthesis by addressing the low draft-token acceptance rates in high-entropy regions. SJD-PAC employs proactive drafting to enhance local acceptance rates and an adaptive continuation mechanism to sustain sequence validation after initial rejections. Experiments on text-to-image benchmarks demonstrate a 3.8x speedup with lossless image quality, highlighting the effectiveness of the proposed optimizations.
Text-to-image synthesis just got almost 4x faster without sacrificing image quality, thanks to a clever twist on Speculative Jacobi Decoding that keeps the generation process moving even when initial drafts are rejected.
Speculative Jacobi Decoding (SJD) offers a draft-model-free approach to accelerate autoregressive text-to-image synthesis. However, the high-entropy nature of visual generation yields low draft-token acceptance rates in complex regions, creating a bottleneck that severely limits overall throughput. To overcome this, we introduce SJD-PAC, an enhanced SJD framework. First, SJD-PAC employs a proactive drafting strategy to improve local acceptance rates in these challenging high-entropy regions. Second, we introduce an adaptive continuation mechanism that sustains sequence validation after an initial rejection, bypassing the need for full resampling. Working in tandem, these optimizations significantly increase the average acceptance length per step, boosting inference speed while strictly preserving the target distribution. Experiments on standard text-to-image benchmarks demonstrate that SJD-PAC achieves a $3.8\times$ speedup with lossless image quality.
