Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Losslessly Compressed Tensors (LCT), a GPU execution framework designed for the repeated encode-decode cycles inherent in training neural networks. LCT leverages Huffman coding with precomputed codebooks and optimized GPU kernels to exploit floating-point data redundancy, achieving compression ratios up to $1.455\times$ with minimal overhead. Experiments show LCT significantly reduces memory usage (e.g., $24.6\%$ for NanoGPT, $23.8\%$ for Qwen3-4B fine-tuning) while incurring only modest increases in step time, enabling larger context lengths on limited hardware.
Train LLMs with up to $1.455\times$ less memory and enable $3.5\times$ longer context lengths by losslessly compressing tensors on-the-fly.
Lossless tensor compression could reduce memory usage during training, but training requires tensors to be repeatedly re-encoded. We introduce Losslessly Compressed Tensors (LCT), a GPU execution framework designed for this repeated encode-decode regime. LCT exploits the natural redundancy in floating point data to reduce storage for weights, activations and optimizer states. Highly optimized GPU kernels implement Huffman coding with precomputed codebooks for low compression and decompression overhead, essential for training. Our analysis characterizes LCT's storage costs and adaptation to magnitude shifts. LCT achieves a compression ratio of up to $1.455\times$, and is demonstrated in training and finetuning settings. On a 162M-parameter NanoGPT, LCT reduces memory usage by $24.6\%$ for a $7.2\%$ step-time increase. In Qwen3-4B fine-tuning, LCT saves up to $23.8\%$ of peak memory, enabling a $3.5\times$ larger context length in training on the same GPU.