Search papers, labs, and topics across Lattice.
The paper introduces Bit-Plane Decomposition Quantization (BPDQ), a novel post-training quantization method for LLMs that constructs a variable quantization grid using bit-planes and scalar coefficients to overcome limitations of fixed-grid quantization at low bitwidths. BPDQ iteratively refines the bit-planes and coefficients using approximate second-order information and progressive error compensation to minimize output discrepancy. Experiments demonstrate that BPDQ enables serving Qwen2.5-72B at 2-bit precision on a single RTX 3090 with minimal accuracy loss (83.85% GSM8K accuracy vs. 90.83% at 16-bit).
Forget fixed uniform intervals: BPDQ unlocks high-fidelity 2-bit quantization for LLMs by adaptively shaping the quantization grid.
Large language model (LLM) inference is often bounded by memory footprint and memory bandwidth in resource-constrained deployments, making quantization a fundamental technique for efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. Fundamentally, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using approximate second-order information while progressively compensating quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85% GSM8K accuracy (vs. 90.83% at 16-bit). Moreover, we provide theoretical analysis showing that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. Code: github.com/KingdalfGoodman/BPDQ.
