Search papers, labs, and topics across Lattice.
This paper introduces SpatioTemporal Adaptive Reward (STAR) Allocation, a novel approach for improving reinforcement learning (RL) post-training in text-to-image generation by addressing the mismatch between scalar rewards and the spatial-temporal structure of generated images. By leveraging text-image attention and constructing dynamic spatial allocation maps, STAR focuses policy updates on the most relevant latent regions, enhancing the model's ability to align generated content with user prompts. Experimental results demonstrate significant improvements in compositional semantic alignment and text rendering across multiple evaluation tasks, achieving high scores without altering the external reward source.
STAR Allocation enables targeted policy updates in text-to-image generation, leading to unprecedented improvements in semantic alignment and text rendering.
Existing RL post-training methods for text-to-image generation usually convert the final-image reward into a single scalar advantage and apply it with the same strength to the entire generative trajectory. However, text-to-image generation naturally has temporal and spatial structure: different denoising steps are responsible for different generation stages, and the content that truly determines text alignment often appears only in part of the image. This granularity mismatch makes it difficult for policy updates to focus on the generative components that actually affect the reward. To address this issue, we propose \textbf{SpatioTemporal Adaptive Reward (STAR) Allocation} for RL post-training of text-to-image diffusion and flow models. STAR uses text-image attention inside the generative model and starts from the core content that the user truly cares about in the prompt. It constructs spatial allocation maps that dynamically vary across denoising steps and rollouts, and allocates the same group-relative advantage to more relevant latent regions with almost no additional computational overhead. STAR then applies stronger policy updates to these regions through a spatially resolved policy objective. We use Stable Diffusion 3.5 Medium as the base model and evaluate on three tasks: GenEval, OCR text rendering, and PickScore. Experimental results show that STAR improves compositional semantic alignment, text rendering, and preference optimization without changing the external reward source, achieving $\mathbf{0.9759}$, $\mathbf{0.9757}$, and $\mathbf{23.60}$ on GenEval, OCR, and PickScore, respectively.
