Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We introduce QuadTok, a visual tokenization framework employing a hierarchical quadtree structure to dynamically allocate representational capacity, improving token efficiency for autoregressive image generation. This approach bridges the gap between 2D spatial binding and 1D sequence flexibility, achieving comparable reconstruction fidelity with fewer tokens on ImageNet and COCO. The resulting GPT-style generative model demonstrates strong performance on ImageNet and enables novel zero-shot spatially controlled generation.
A quadtree visual tokenizer slashes token count for autoregressive image generation by adaptively focusing representational power on intricate image regions.
We introduce QuadTok, a novel framework for visual tokenization and autoregressive image generation. Compared to traditional approaches using 2D grids or 1D token sequences, we propose a hierarchical quadtree structure, bridging the gap between 2D spatial binding and 1D sequence-level flexibility. The QuadTok tokenizer dynamically allocates representational capacity to visually intricate areas while leaving homogeneous regions at a coarse resolution. Compared with a fixed 256-token grid, our ImageNet-trained tokenizer saves approximately 10% of tokens on ImageNet and 9% when transferred zero-shot to the COCO dataset, while maintaining comparable reconstruction fidelity. Furthermore, the natural causality introduced by the tree structure seamlessly enables autoregressive image generation. Conditioned on a quadtree topology supplied before generation, our 947M GPT-style generative model achieves a 2.08 gFID on the ImageNet 256 times 256 benchmark. Additionally, leveraging the strong spatial correlation preserved by the quadtree structure, the QuadTok generator enables zero-shot spatially controlled image generation capabilities. Code: https://github.com/myc634/QuadTok.