Search papers, labs, and topics across Lattice.
This paper introduces PiSO (Piecewise Scale Optimization), an innovative algorithm that optimally computes channel-wise weight scales for post-training quantization (PTQ) using calibration data, rather than relying on simplistic heuristics. By partitioning the scale search space into intervals with closed-form minimizers, PiSO enhances the quantization process, particularly in challenging low-bit scenarios. Experimental results on Llama and Qwen models reveal significant improvements in perplexity and zero-shot accuracy, especially as the target bit-width decreases, highlighting the method's effectiveness in optimizing quantization performance.
PiSO achieves remarkable gains in quantization accuracy, revealing that careful scale optimization can dramatically enhance model performance even in low-bit regimes.
Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations. The scaling factor that defines the quantization grid is typically chosen using simple, data-free heuristics. In this work, we present PiSO (Piecewise Scale Optimization), an algorithm that leverages calibration data to compute the optimal channel-wise weight scales exactly and efficiently under round-to-nearest quantization. PiSO partitions the scale search space into finitely many intervals on which the objective admits a closed-form minimizer. We extend PiSO to group-wise quantization via principled heuristics and propose effective strategies for interleaving scale optimization with error correction. Experiments on Llama and Qwen models across multiple model sizes and target weight bit-widths demonstrate consistent improvements in perplexity and downstream zero-shot accuracy, both standalone and combined with error correction. In particular, we observe increased benefits as the target bit-width narrows and quantization becomes more challenging.
