Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SpectrumQuant, a novel post-training quantization method for LLMs that leverages spectral alignment via covariance maximization to construct data-aware orthogonal transforms. By reformulating rotation selection from first principles, it identifies a principled criterion for designing quantization-friendly transforms that align activation spectra. Experiments demonstrate SpectrumQuant achieves competitive performance with state-of-the-art methods while significantly reducing memory and time overhead compared to learned rotation approaches.
A closed-form, covariance-aware orthogonal transform for LLM quantization achieves state-of-the-art performance with a third of the memory and time of learned rotation methods.
Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing covariance leads to spectral alignment and provides a principled criterion for constructing quantization-friendly orthogonal transforms. Based on this insight, we propose SpectrumQuant, a closed-form rotation method that estimates activation covariance from calibration data and uses it to construct a covariance-aware orthogonal transform for post-training quantization. Our formulation characterizes quantization-friendly rotations as compositions of alignment and mixing components, while recovering Hadamard matrices as a special case. Interestingly, a class of butterfly matrices belongs to this category. In the realm of LLM quantization, SpectrumQuant bridges the gap between data-agnostic structured rotations and learned rotations, providing an effective and efficient data-aware alternative for post-training quantization. Experiments on Llama and Mistral models show that SpectrumQuant is competitive with state-of-the-art rotation strategies while requiring less than one-third of the GPU memory and significantly less time than learned rotations approaches like SpinQuant and DartQuant.