Search papers, labs, and topics across Lattice.
The paper introduces CaTok, a 1D causal image tokenizer with a MeanFlow decoder, designed to address the limitations of existing visual tokenizers in autoregressive models. CaTok selects tokens over time intervals and binds them to the MeanFlow objective, learning causal 1D representations that support both fast one-step generation and high-fidelity multi-step sampling. The method incorporates a regularization technique, REPA-A, to align encoder features with Vision Foundation Models, improving training stability and speed.
CaTok achieves state-of-the-art ImageNet reconstruction with fewer training epochs by learning causal 1D image representations, outperforming existing visual tokenizers.
Autoregressive (AR) language models rely on causal tokenization, but extending this paradigm to vision remains non-trivial. Current visual tokenizers either flatten 2D patches into non-causal sequences or enforce heuristic orderings that misalign with the"next-token prediction"pattern. Recent diffusion autoencoders similarly fall short: conditioning the decoder on all tokens lacks causality, while applying nested dropout mechanism introduces imbalance. To address these challenges, we present CaTok, a 1D causal image tokenizer with a MeanFlow decoder. By selecting tokens over time intervals and binding them to the MeanFlow objective, as illustrated in Fig. 1, CaTok learns causal 1D representations that support both fast one-step generation and high-fidelity multi-step sampling, while naturally capturing diverse visual concepts across token intervals. To further stabilize and accelerate training, we propose a straightforward regularization REPA-A, which aligns encoder features with Vision Foundation Models (VFMs). Experiments demonstrate that CaTok achieves state-of-the-art results on ImageNet reconstruction, reaching 0.75 FID, 22.53 PSNR and 0.674 SSIM with fewer training epochs, and the AR model attains performance comparable to leading approaches.
