Search papers, labs, and topics across Lattice.
This paper introduces an end-to-end training pipeline for autoregressive image generation that jointly optimizes a 1D semantic tokenizer and the generative model, allowing direct supervision of the tokenizer from generation results. They explore leveraging vision foundation models to enhance the 1D tokenizer. The resulting autoregressive model achieves a state-of-the-art FID score of 1.48 on ImageNet 256x256 without guidance, demonstrating the effectiveness of joint training.
Jointly training the tokenizer and autoregressive model slashes ImageNet FID to 1.48, finally making end-to-end autoregressive image generation competitive.
Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256x256 generation.
