Search papers, labs, and topics across Lattice.
The paper introduces AViTS, an adaptive spatiotemporal token selection framework designed to enhance the efficiency of Dynamic-Resolution Diffusion Transformers (DiTs) by optimizing the upsampling process. By leveraging latent-text attention to model spatial importance and analyzing token-level feature variation for temporal importance, AViTS selectively refines only the most critical tokens during resolution transitions. This approach significantly reduces computational costs, achieving up to 6.34x efficiency gains and nearly 9x reduction in FLOPs while maintaining high-quality generation outputs.
AViTS reduces redundant computation in dynamic-resolution generation by up to 14.76x while preserving fine-detail consistency in outputs.
Diffusion Transformers (DiTs) achieve high-quality generation but are costly due to iterative sampling. Dynamic-resolution sampling reduces early-stage cost by denoising at low resolution; however, uniformly upsampling all latent tokens at resolution transitions incurs redundant computation and may degrade fine-detail consistency. Existing partial upsampling strategies typically rely on local latent structure cues or single-step statistics, making it difficult to jointly capture token-text semantic relevance and token-wise representation dynamics across diffusion steps. We propose AViTS, an adaptive spatiotemporal token selection framework for dynamic-resolution DiTs. AViTS models spatial importance via latent-text attention and temporal importance via token-level feature variation across diffusion timesteps, and fuses them to enable spatiotemporal importance-aware selective upsampling: it prioritizes resolution refinement for critical tokens while deferring less important ones, thereby reducing redundant high-resolution computation and improving the quality-efficiency trade-off. AViTS achieves up to 6.34x on FLUX and nearly 9x FLOPs reduction on Qwen-Image-Edit and FLUX.1-Kontext-dev, orthogonal to distillation, quantization, and feature caching, and reaching 14.76x with distilled models. Code: https://github.com/QHR69/AViTS
