Search papers, labs, and topics across Lattice.
This paper introduces CCCL, a compression-based collective communication library designed to reduce communication overhead in LLM workloads without requiring user-side code modifications. CCCL fuses compression kernels to minimize memory accesses and integrates with NCCL to eliminate data coalescing. Evaluation shows CCCL achieves up to 3x NVLink bandwidth, improving end-to-end throughput in vLLM PD disaggregation workloads by up to 10.1% and microbenchmark throughput by up to 30%.
Get up to 10% more throughput on your LLM disaggregation workloads just by swapping in this drop-in collective communications library with built-in compression.
Collective communication incurs significant overhead in LLM workloads. Although overlapping communication with computation in application-level is a common strategy, it often requires substantial code modifications and is impractical for many workloads (e.g., tensor and expert parallelism). We present CCCL, a built-in compression-based collective communication library that supports operations such as allreduce, alltoall, and send/recv without requiring any user-side changes, thereby enabling seamless adoption in existing applications. CCCL tightly fuses compression kernels to minimize memory accesses and integrates with NCCL to eliminate the data coalescing stage, making it fast enough (up to 3x NVLink bandwidth) to sustain communication. Our evaluation shows that CCCL improves end-to-end throughput in vLLM PD disaggregation workloads by up to 10.1% and microbenchmark throughput by up to 30%.
