Search papers, labs, and topics across Lattice.
The paper introduces Patch Forcing (PF), a framework for diffusion and flow-based image generation that uses patch-level noise scales and adaptive sampling to focus compute on difficult regions. PF employs a timestep sampler that controls the maximum patch-level information during training and a per-patch difficulty head to dynamically allocate compute. Experiments on class-conditional ImageNet and text-to-image synthesis demonstrate that PF achieves superior results by advancing easier regions earlier to provide context for harder ones.
By intelligently focusing compute on the most challenging image regions, Patch Forcing significantly boosts image generation quality without adding computational overhead.
Diffusion- and flow-based models usually allocate compute uniformly across space, updating all patches with the same timestep and number of function evaluations. While convenient, this ignores the heterogeneity of natural images: some regions are easy to denoise, whereas others benefit from more refinement or additional context. Motivated by this, we explore patch-level noise scales for image synthesis. We find that naively varying timesteps across image tokens performs poorly, as it exposes the model to overly informative training states that do not occur at inference. We therefore introduce a timestep sampler that explicitly controls the maximum patch-level information available during training, and show that moving from global to patch-level timesteps already improves image generation over standard baselines. By further augmenting the model with a lightweight per-patch difficulty head, we enable adaptive samplers that allocate compute dynamically where it is most needed. Combined with noise levels varying over both space and diffusion time, this yields Patch Forcing (PF), a framework that advances easier regions earlier so they can provide context for harder ones. PF achieves superior results on class-conditional ImageNet, remains orthogonal to representation alignment and guidance methods, and scales to text-to-image synthesis. Our results suggest that patch-level denoising schedules provide a promising foundation for adaptive image generation.
