Search papers, labs, and topics across Lattice.
This paper introduces Disk-Backed Parallel Pull (DBPP), a novel method for efficiently downloading large container images by writing data directly to disk rather than relying on in-memory reassembly. This approach mitigates out-of-memory (OOM) issues on GPU nodes while maintaining high throughput, achieving a peak memory reduction of 8.7–25.3 times compared to the existing containerd 2.2 method. The findings suggest that DBPP not only optimizes container image pulls but also has broader implications for data pipelines that require ordering without excessive memory usage.
DBPP slashes peak memory usage by up to 25 times during large container image pulls, preventing OOM failures on GPU nodes.
AI/ML workloads increasingly run as containers, where a container image must be downloaded to the host before the workload can start. This cold image pull lands on the critical path whenever a training or inference job scales up or a host is updated, and for GPU workloads it has become the dominant component of startup time as AI/ML images reach 31--48~GiB compressed. We present Disk-Backed Parallel Pull (DBPP), an alternative to the in-memory ordered reassembly used by containerd~2.2, the upstream container runtime. containerd splits layers into chunks fetched concurrently over HTTP range requests, but chunks that arrive out of order accumulate in the runtime heap until a sequential consumer drains them in order. This backlog grows with image size, and on GPU nodes where host memory is shared with frameworks and model weights, it leads to out of memory (OOM) termination of the runtime itself. DBPP writes each chunk directly to its target byte offset on disk, eliminating the ordering dependency and bounding memory regardless of image size. Because each layer lands on disk as a complete, seekable file, DBPP runs SHA-256 digest verification and decompression simultaneously, two passes containerd must run one after the other. In controlled experiments across five production-scale images (up to 48.5~GiB), DBPP reduces peak daemon memory by 8.7--25.3$\times$ while maintaining comparable pull throughput. On a memory-constrained node, containerd~2.2 is OOM-killed pulling a 31.4~GiB image while DBPP completes the same pull. The underlying idea reaches past container images: any pipeline that buffers data in memory only to enforce ordering can move that buffer to disk once the backing store is fast enough, trading a scarce, contended resource for an abundant one.
Amazon Science