Search papers, labs, and topics across Lattice.
This paper investigates the disconnect between visual representation quality and generative utility in pixel diffusion transformers, finding that external semantic alignment doesn't guarantee generative success and architectural separation offers limited gains. To address this, they propose Representation Guidance (RG), which uses a low-dimensional bottleneck to force generation-relevant information into a compact representation that actively guides decoder synthesis via flow-matching target modification. RG achieves state-of-the-art FID scores on ImageNet 256x256 and 512x512, and demonstrates strong performance in text-to-image generation.
Generative representations need more than just semantic alignment; they require active exploitation, as shown by Representation Guidance outperforming larger models on ImageNet with a novel flow-matching target modification.
Visual representations are largely developed for understanding, yet what makes them useful for generation remains unclear. We study this question in pixel diffusion transformers and uncover two key insights. First, better representation quality does not necessarily translate into greater generative utility, as externally aligned semantics can be progressively lost during pixel synthesis. Second, architectural separation with specialized pixel decoders mainly redistributes semantic and spatial modeling, without necessarily yielding representations better suited to generation. These insights suggest that a useful generative representation must be both properly formed and actively exploited. Motivated by these insights, we introduce Representation Guidance (RG), which directly forms and exploits internal representations while retaining a plain Transformer architecture. A low-dimensional bottleneck forces generation-relevant information into a compact representation, which is then explicitly used to guide decoder generation by modifying the flow-matching target according to its induced prediction offset. RG learns clearer semantic and spatial structure while scaling efficiently in both parameters and computation. On ImageNet 256x256, RG-L/16 with only 459M parameters outperforms substantially larger pixel diffusion transformers, while RG-XL/16 achieves an FID of 1.55. RG further reaches an FID of 1.74 at 512x512 and extends to text-to-image generation, achieving 0.89 on GenEval and 87.1 on DPG-Bench.
