Search papers, labs, and topics across Lattice.
This paper introduces DiT-Reward, a method that transforms a pretrained text-to-image Diffusion Transformer into a reward model for evaluating generated images by processing near-clean image latents and aggregating text-conditioned representations. The results demonstrate that DiT-Reward surpasses the existing HPSv3 model across all four preference benchmarks, achieving notable scores of 85.6% on HPDv2 and 77.6% on HPDv3, while also providing a significant inference speedup. Furthermore, the findings indicate that the middle-to-late transformer layers are crucial for optimal reward performance, highlighting the effectiveness of combining representations from various stages of the model.
DiT-Reward not only outperforms existing models in image evaluation but also accelerates inference by 1.65x without sacrificing quality.
Can representations learned for image generation also support the evaluation of generated images? We study text-to-image reward prediction as a downstream task of generative representation learning. To this end, we introduce DiT-Reward, which converts a pretrained text-to-image Diffusion Transformer into a reward model by processing near-clean image latents and aggregating text-conditioned image representations across transformer layers. Under the same training data mixture as HPSv3, DiT-Reward outperforms HPSv3 on all four evaluated preference benchmarks, reaching 85.6% on HPDv2 and 77.6% on HPDv3. When the generative backbone is frozen, a lightweight learned head can still extract meaningful preference predictions from its representations. Probing across depth further reveals that downstream reward performance is strongest in the middle-to-late layers and benefits from combining representations across different stages. We also observe consistent positive scaling with generative backbone capacity. Finally, when used to optimize Stable Diffusion 3.5 Large with Flow-GRPO, DiT-Reward outperforms HPSv3 along the matched training trajectory, with particularly clear gains in realism. Direct latent scoring also achieves a 1.65x inference speedup over HPSv3 with comparable peak memory. These results show that pretrained generative DiTs provide transferable representations for reward modeling and policy optimization.
Tsinghua AI