Search papers, labs, and topics across Lattice.
EvoTok is introduced as a unified image tokenizer that bridges the granularity gap between visual understanding and generation by encoding images into a cascaded sequence of residual tokens via residual vector quantization. This approach creates an evolution trajectory within a shared latent space, where early stages capture low-level details and deeper stages transition to high-level semantics. EvoTok achieves strong reconstruction quality (0.43 rFID on ImageNet-1K at 256x256) with only 13M training images and demonstrates promising performance on both visual understanding and image generation benchmarks when integrated with a large language model.
Forget billion-scale datasets: EvoTok achieves state-of-the-art image tokenization for both understanding and generation using a residual evolution process trained on just 13M images.
The development of unified multimodal large language models (MLLMs) is fundamentally challenged by the granularity gap between visual understanding and generation: understanding requires high-level semantic abstractions, while image generation demands fine-grained pixel-level representations. Existing approaches usually enforce the two supervision on the same set of representation or decouple these two supervision on separate feature spaces, leading to interference and inconsistency, respectively. In this work, we propose EvoTok, a unified image tokenizer that reconciles these requirements through a residual evolution process within a shared latent space. Instead of maintaining separate token spaces for pixels and semantics, EvoTok encodes an image into a cascaded sequence of residual tokens via residual vector quantization. This residual sequence forms an evolution trajectory where earlier stages capture low-level details and deeper stages progressively transition toward high-level semantic representations. Despite being trained on a relatively modest dataset of 13M images, far smaller than the billion-scale datasets used by many previous unified tokenizers, EvoTok achieves a strong reconstruction quality of 0.43 rFID on ImageNet-1K at 256x256 resolution. When integrated with a large language model, EvoTok shows promising performance across 7 out of 9 visual understanding benchmarks, and remarkable results on image generation benchmarks such as GenEval and GenAI-Bench. These results demonstrate that modeling visual representations as an evolving trajectory provides an effective and principled solution for unifying visual understanding and generation.
