Search papers, labs, and topics across Lattice.
This paper investigates the role of explicit timestep embeddings in diffusion models, revealing that they may not be necessary for effective denoising. By providing both empirical evidence and a theoretical framework, the authors demonstrate that U-Net and Diffusion Transformer architectures can achieve optimal performance without these embeddings, as shown through extensive ablation studies on the CelebA and CIFAR-10 datasets. The results indicate that time-agnostic models not only maintain high structural fidelity but can also outperform their conditioned counterparts in key metrics like FID, precision, and recall.
Removing timestep embeddings from diffusion models can lead to surprising improvements in performance, challenging long-held assumptions about their necessity.
Diffusion models rely heavily on explicit timestep embeddings to modulate the denoising process across various noise scales. In this work, we challenge the necessity of these temporal signals by analyzing their impact on U-Net and Diffusion Transformer architectures. Beyond empirical evidence, we provide a theoretical framework demonstrating that, under certain conditions, the global minimizer of the diffusion training objective can be achieved without explicit timestep conditioning. Our findings reveal a surprising robustness when timestep embeddings are completely removed. Extensive ablation studies on the CelebA and CIFAR-10 datasets show that these time-agnostic models can maintain high structural fidelity and even surpass their conditioned counterparts in competitive metrics, including FID, precision, and recall. Our analysis suggests these architectures can implicitly infer noise scales from the corrupted input under specific assumptions, rendering explicit temporal conditioning redundant. This study challenges long-standing temporal conditioning paradigms and paves the way for more efficient and structurally focused generative architectures.
