Search papers, labs, and topics across Lattice.
The paper introduces SemanticDialect, a novel quantization method for video Diffusion Transformers (DiT) that addresses the challenge of maintaining video quality under high activation variation and semantic/temporal coherence requirements. SemanticDialect extends block-wise mixed-format quantization by using lookup tables to efficiently select the optimal quantization format per block and employs activation decomposition with attention-guided salient token selection to reduce quantization error. Experiments on VDiT models demonstrate that SemanticDialect outperforms existing quantization techniques and approaches FP16 precision on Open-Sora 2.0.
Achieve near-FP16 video diffusion transformer quality with a quantization method that intelligently adapts to semantic content, outperforming prior quantization techniques.
Diffusion Transformers (DiT) achieve strong video generation quality, but their memory and compute costs hinder edge deployment. Quantization can reduce these costs, yet existing methods often degrade video quality under high activation variation and the need to preserve semantic/temporal coherence. We propose SemanticDialect, which advances recent block-wise mixed-format quantization-selecting a per-block optimal format (a dialect) from multiple candidates (a formatbook)-by scaling the formatbook with lookup tables for quantization error and quantized values, enabling efficient per-block format selection and quantization at low online cost. We also introduce activation decomposition that reduces quantization error by re-quantizing and adding back residual errors, with attention-guided salient token selection. We further propose semantic-aware dialect assignment (SeDA) to improve quantized value consistency by sharing a sub-formatbook among semantically correlated tokens. Experiments on video DiT (VDiT) models show that SemanticDialect outperforms prior VDiT quantization methods and fine-grained block-wise format baselines, while approaching FP16 quality on Open-Sora 2.0.
