Search papers, labs, and topics across Lattice.
VidaForge establishes an open, fully traceable five-stage execution pipeline and a 3.14-million-clip dataset to systematically benchmark video data curation strategies. Addressing the lack of transparent infrastructure in video pretraining, the authors evaluated recipes varying in visual quality and coverage during early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1. Their empirical tests show that broader dataset coverage outperforms aggressive quality filtering on downstream tasks across both generative and self-supervised objectives, directly contradicting what pretraining loss curves suggest.
Pretraining loss actively misleads video data curation: broad data coverage systematically beats aggressive quality filtering across downstream benchmarks in both Wan 2.1 and V-JEPA 2.1.
Video foundation models increasingly rely on large-scale pretraining data, yet the end-to-end data pipelines behind them remain largely closed and difficult to inspect or reuse. Researchers seeking to understand how video data recipes affect model pretraining often need to build substantial infrastructure before testing even a focused hypothesis. We present VIDAFORGE, an open research infrastructure that represents a video data recipe as an executable five-stage workflow from raw videos to training datasets. A decision in this workflow can be varied to construct alternative datasets while preserving how every sample was produced. To demon strate this research workflow, we compare data recipes with different coverage and quality in early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1. Across both learning objectives, the broader-coverage recipe achieves the highest downstream benchmark scores, while loss-based evaluation favors different recipes. This study demonstrates how VidaForge connects data-recipe choices to downstream model performance. We further release VIDAFORGE-3M, containing 3.14 million scene level clips totaling 6,475 hours, with fine-grained annotations and curation signals for video data-recipe research.
