Search papers, labs, and topics across Lattice.
This paper introduces LLMCodec, a novel method that leverages video codecs for the efficient weight compression of large language models (LLMs). By integrating affine quantization with the VVC/H.266 video codec, LLMCodec addresses the limitations of traditional compression methods that require fine-tuning or calibration data. The results show that LLMCodec significantly enhances performance, achieving a 1.5x reduction in perplexity and a 21% improvement in downstream task accuracy on the LLaMA-3-8B model at 2-bit precision.
Video codecs can slash LLM perplexity by over 1.5x while boosting task accuracy by 21%, revolutionizing model compression strategies.
The rapid development of large language models(LLMs) has led to remarkable advances in natural language processing. However, the increasing scale of these models introduces substantial challenges in terms of storage, transmission, and deployment. Though great efforts have been devoted to model compression and quantization, existing methods often rely on fine-tuning or calibration data, which exhibit limited generalization across different tensor types. In this paper, we argue that video codecs offer a promising solution for LLM compression, due to their inherent compatibility with matrix structured data, configurable compression strategies, and the availability of highly optimized, off-the-shelf implementations. Therefore, we present LLMCodec, a video codec-based LLM compression method that integrates affine quantization with the recent VVC/H.266 video codec. Beyond VVC, we further compare a range of video codecs and encoding profiles to evaluate their impact on compression performance. Experiments on different models demonstrate the robustness and generality of LLMCodec. Notably, on LLaMA-3-8B at 2-bit precision, LLMCodec reduces perplexity by over 1.5x and improves downstream task accuracy by 21% compared with the existing method.
