Search papers, labs, and topics across Lattice.
This paper introduces Arachne, a novel training framework designed to enhance the efficiency of large-scale Text-to-Video (T2V) model training by addressing the challenges posed by data heterogeneity and workload imbalances. By decomposing the training process into fine-grained computational units called cascades, Arachne orchestrates distributed execution and synchronization, significantly improving resource utilization. The results show that Arachne can reduce iteration time by up to 65% compared to existing frameworks, with performance benefits increasing as the scale of training grows.
Arachne achieves up to 65% faster iteration times for Text-to-Video model training by optimizing the orchestration of computational units across diverse data.
The rising demand for AI-generated videos is fueled by advances in large-scale Text-to-Video (T2V) models, trained on extensive datasets of video clips spanning diverse resolutions and durations. To address this data heterogeneity, current training methods often use a bucketing strategy that groups samples into discrete buckets for efficiency. However, this approach struggles to scale with compute and data volumes under static parallelism schemes, such as data and sequence parallelism, leading to significant workload imbalances and hardware under-utilization. In this paper, we present Arachne, a novel training framework for efficient T2V model training at scale. Arachne decomposes the training process into fine-grained computational units, called \textit{cascades}, orchestrating their distributed execution and synchronization across the cluster through coordinated spatial and temporal optimization. Our comprehensive evaluation demonstrates that Arachne reduces iteration time by up to 65\% over leading frameworks, exhibiting a positive scaling trend where its performance advantages amplify as training scale grows.
