Search papers, labs, and topics across Lattice.
This paper introduces LoRaQ, a novel approach to post-training quantization that optimizes low-rank approximation for 4-bit quantization, addressing the performance degradation typically associated with aggressive quantization of large diffusion transformers. By eliminating the need for high-precision auxiliary branches and employing a data-free calibration method, LoRaQ enables a fully sub-16 bit quantization pipeline while effectively compensating for quantization errors. Experimental results show that LoRaQ surpasses existing state-of-the-art methods on benchmark datasets like Pixart-$危$ and SANA, particularly in mixed-precision configurations, demonstrating its practical viability for resource-constrained hardware.
LoRaQ achieves superior generative performance with fully quantized architectures, challenging the necessity of high-precision branches in low-rank approximation.
Post-training quantization (PTQ) is essential for deploying large diffusion transformers on resource-constrained hardware, but aggressive 4-bit quantization significantly degrades generative performance. Low-rank approximation methods have emerged as a promising solution by appending auxiliary linear branches to restore performance. However, current state-of-the-art approaches assume these branches must retain high precision (W16A16) and rely on heavy, data-dependent calibration for initialization. We challenge both limitations with LoRaQ (Low-Rank Approximated Quantization), a simple, data-free calibration approach that optimizes quantization error compensation. By overcoming the need for high-precision branches, LoRaQ enables the first fully sub-16 bit pipeline, allowing the low-rank branch itself to be quantized. We demonstrate that, at equal memory overhead, LoRaQ outperforms the state-of-the-art methods in their native implementations on Pixart-$危$ and SANA. We also analyze mixed-precision configurations, showing that setups such as W8A8, W6A6, and W4A8 for the low-rank branch, alongside a W4 main layer, yield superior results while maintaining a fully quantized architecture compatible with modern mixed-precision hardware.
