Search papers, labs, and topics across Lattice.
EdgeDiT is introduced as a hardware-aware optimization framework for Diffusion Transformers (DiT) to enable efficient on-device image generation on mobile Neural Processing Units (NPUs). The framework identifies and prunes structural redundancies within the DiT backbone that are taxing for mobile data-flows. EdgeDiT achieves a 20-30% reduction in parameters, a 36-46% decrease in FLOPs, and a 1.65-fold reduction in on-device latency while maintaining image quality, offering a superior FID/latency trade-off compared to mobile U-Nets and vanilla DiT variants.
Runaway compute costs for diffusion models on GPUs? EdgeDiT slashes parameters by 30% and latency by 40% while maintaining image quality, all on your phone.
Diffusion Transformers (DiT) have established a new state-of-the-art in high-fidelity image synthesis; however, their massive computational complexity and memory requirements hinder local deployment on resource-constrained edge devices. In this paper, we introduce EdgeDiT, a family of hardware-efficient generative transformers specifically engineered for mobile Neural Processing Units (NPUs), such as the Qualcomm Hexagon and Apple Neural Engine (ANE). By leveraging a hardware-aware optimization framework, we systematically identify and prune structural redundancies within the DiT backbone that are particularly taxing for mobile data-flows. Our approach yields a series of lightweight models that achieve a 20-30% reduction in parameters, a 36-46% decrease in FLOPs, and a 1.65-fold reduction in on-device latency without sacrificing the scaling advantages or the expressive capacity of the original transformer architecture. Extensive benchmarking demonstrates that EdgeDiT offers a superior Pareto-optimal trade-off between Frechet Inception Distance (FID) and inference latency compared to both optimized mobile U-Nets and vanilla DiT variants. By enabling responsive, private, and offline generative AI directly on-device, EdgeDiT provides a scalable blueprint for transitioning large-scale foundation models from high-end GPUs to the palm of the user.
