Search papers, labs, and topics across Lattice.
This paper introduces SemTok, a novel visual tokenizer that compresses 2D images into 1D discrete tokens, aiming to capture compact global semantics beyond pixel-level restoration. SemTok employs a 2D-to-1D tokenization scheme, semantic alignment constraints, and a two-stage generative training strategy to achieve state-of-the-art image reconstruction fidelity with compact token representations. Experiments demonstrate that SemTok improves downstream image generation tasks when used within a masked autoregressive generation framework.
Compressing images into 1D token sequences can yield state-of-the-art reconstruction fidelity, challenging the necessity of 2D spatial grids for visual tokenization.
Visual generative models based on latent space have achieved great success, underscoring the significance of visual tokenization. Mapping images to latents boosts efficiency and enables multimodal alignment for scaling up in downstream tasks. Existing visual tokenizers primarily map images into fixed 2D spatial grids and focus on pixel-level restoration, which hinders the capture of representations with compact global semantics. To address these issues, we propose \textbf{SemTok}, a semantic one-dimensional tokenizer that compresses 2D images into 1D discrete tokens with high-level semantics. SemTok sets a new state-of-the-art in image reconstruction, achieving superior fidelity with a remarkably compact token representation. This is achieved via a synergistic framework with three key innovations: a 2D-to-1D tokenization scheme, a semantic alignment constraint, and a two-stage generative training strategy. Building on SemTok, we construct a masked autoregressive generation framework, which yields notable improvements in downstream image generation tasks. Experiments confirm the effectiveness of our semantic 1D tokenization. Our code will be open-sourced.
