Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Ideal, an In-depth Alignment framework that enhances discrete representation autoencoders by aligning quantized tokens with both shallow and deep features from pretrained vision foundation models (VFMs). The motivation stems from the observation that shallow VFM features retain richer local details, which are crucial for high-quality image reconstruction, especially after discretization. Experimental results show that Ideal significantly improves reconstruction performance, achieving a 0.61 rFID on ImageNet and setting a new state-of-the-art gFID of 1.89 for autoregressive image generation, outperforming previous methods by a notable margin.
Aligning shallow and deep features in representation autoencoders leads to a dramatic improvement in image reconstruction quality, setting new benchmarks in the field.
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail. This limitation becomes even more severe after discretization, where missing low-level information is difficult to recover. In fact, we observe that shallow VFM features retain considerably richer local appearance and structural detail, which complements the high-level semantics carried by deep features used in existing RAEs. Motivated by this complementary property, we propose Ideal, an In-depth Alignment framework for discrete representation autoencoding. By jointly aligning quantized tokens with both shallow and deep VFM features, Ideal enables the resulting discrete visual tokens to preserve both visual fidelity and rich semantics. Extensive experiments demonstrate that Ideal yields superior reconstruction performance, achieving 0.61 rFID on ImageNet and outperforming the previous best method by 0.28. When used for autoregressive image generation, Ideal further produces a gFID of 1.89, establishing a new state of the art for autoregressive image generation.