Search papers, labs, and topics across Lattice.
This paper introduces BiSCo-LLM, a novel binary spherical coding framework designed for extreme low-bit compression of large language model weights without the need for lookup tables. By mapping weight chunks onto a unit hypersphere and employing a residual encoding strategy, the method achieves efficient representation while minimizing reconstruction error. The results demonstrate significant improvements in model performance and storage efficiency, making it feasible to deploy LLMs in resource-constrained environments.
Achieving extreme low-bit compression without lookup tables could revolutionize how we deploy large language models in memory-limited settings.
Large language models (LLMs) are increasingly constrained by memory capacity, weight bandwidth, and checkpoint storage during deployment. Existing low-bit compression methods mainly follow two directions. Scalar or group-wise quantization is simple and compatible with efficient low-precision kernels, but its representation capacity becomes limited when the target budget approaches 2 bits per weight. Vector-quantized weight compression provides a richer block-level representation, but usually introduces explicit codebooks, index lookup, and additional storage accounting. This paper presents BiSCo-LLM, a codebook-free binary spherical coding framework for extreme low-bit LLM weight compression. The core pipeline is built on three components. First, local weight chunks are mapped onto a unit hypersphere and binarized into compact spherical codes, so that the main payload is a bit-packed sign stream rather than explicit VQ centroids. Second, a residual BSQ stage encodes the reconstruction error left by the base spherical codec, providing an explicit rate-distortion path without stored codebooks. Third, category-wise recovery distillation is performed after replacing each Transformer module category, reducing the mismatch between local weight reconstruction and assembled model behavior. A small 8-bit protected-channel path is used as an auxiliary stabilization mechanism for sensitive channels and is counted separately from the BSQ payload. The reported storage budget includes binary codes, neural decoders, protected-channel payloads, LoRA adapters, and metadata.
