Search papers, labs, and topics across Lattice.
This paper introduces FlowTrain, a novel flow-based decoupled training framework for vision-language models (VLMs) that enhances training efficiency by reformulating the process as a producer-consumer dataflow with a unified memory pool. By allowing the encoder and backbone to progress independently and employing a heterogeneous parallel allocator, FlowTrain optimizes allocation and scheduling based on module-specific strategies. Experimental results demonstrate that FlowTrain achieves over 50% model-fine-tuning utilization (MFU) and up to 1.7x throughput improvement, significantly closing the efficiency gap with unimodal language models (LLMs).
FlowTrain redefines VLM training efficiency, achieving up to 1.7x throughput improvements by decoupling execution and optimizing resource allocation.
Industrial-grade distributed training of vision-language models (VLMs) remains far less efficient than that of unimodal LLMs. Existing solutions either follow a monolithic design that assigns uniform parallelism to heterogeneous modules or adopt a disaggregated deployment that separates modules while executing them as a batch-synchronized pipeline. In this paper, we highlight that the above solutions are still not sufficient, and VLM training can be further decoupled. To this end, we present FlowTrain, a flow-based decoupled training framework that reformulates VLM training as a producer-consumer dataflow coordinated through a unified memory pool. The encoder and backbone can progress independently over a global virtual address space. Since this execution decoupling fundamentally changes the optimization objective of allocation and scheduling, FlowTrain further introduces a heterogeneous parallel allocator that assigns module-specific parallelism strategies by solving a throughput matching problem. The dynamic packing scheduler is used to construct balanced microbatches at runtime according to the actual LLM-side computation cost. Extensive experiments on real-world workloads show that FlowTrain achieves over 50% MFU and up to 1.7x throughput improvement, narrowing the efficiency gap to LLM-only training.
