Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the growing CI bottleneck in LLM training frameworks by introducing FastCI, a GPU-efficient testing framework. FastCI utilizes runtime evidence to intelligently select and prioritize tests, pruning redundant executions and optimizing workloads beyond validation scope. The proposed system achieves a 77.5% reduction in CI latency and a 63.9% decrease in GPU resource usage while maintaining high code coverage.
LLM training CI is now 77.5% faster and 63.9% more GPU-efficient by intelligently pruning and prioritizing tests based on runtime evidence.
As large language models (LLMs) keep growing in size and complexity, their training frameworks evolve at a rapid pace as well. Therefore, continuous integration (CI) is critical for maintaining the quality and stability of these frameworks. However, unlike traditional software, CI for LLM training frameworks relies on GPU-intensive tests, which usually involve complete model training or evaluation. This leads CI itself to become a new bottleneck for fast-paced development. In this paper, we introduce FastCI, a framework that improves the efficiency of CI for LLM training frameworks. FastCI leverages runtime evidence to select affected tests and prune tests that execute changed code in equivalent contexts. Then FastCI prioritizes high-risk tests to expose potential failures earlier, and optimizes test workloads along dimensions outside the intended validation scope of each test. Evaluated on the CI workload of our LLM training framework, FastCI reduces the CI latency by 77.5% and the GPU resource usage by 63.9%, while improving the modified code coverage retention by 3.2%, compared with the currently deployed CI pipelines. FastCI has now been integrated into the CI pipelines of our LLM training framework at ByteDance.