Search papers, labs, and topics across Lattice.
Multimodal Flow introduces a novel fully continuous generative framework for language and vision by modeling them as ordered continuous hyperchunks within a shared embedding space, thereby avoiding the quantization bottleneck and modality-specific objectives of prior unified models. This approach utilizes a chunk-causal flow backbone trained via Flow Matching to learn a single vector field for joint cross-modal generation, enabling competitive performance on multimodal benchmarks with significantly less pretraining data. The success of Multimodal Flow establishes continuous chunk-based embedding flow modeling as a promising new paradigm for unified multimodal AI.
A single continuous flow model now unifies language and vision generation, outperforming hybrid and discrete models under matched budgets and achieving strong benchmarks with drastically less data.
We present Multimodal Flow, a fully continuous generative model of language and vision. Most unified multimodal models either model both language and quantized images as discrete tokens or combine discrete language prediction with continuous image generation. The former introduces a visual quantization bottleneck. The latter requires modality-dependent objectives and sampling procedures. Fully continuous modeling avoids these trade-offs and enables a shared generative process, but remains underexplored for multimodal pretraining. Multimodal Flow introduces a unified continuous architecture that integrates multimodal continuous representations with a shared chunk-causal flow backbone. It organizes text blocks and images as ordered continuous hyperchunks, preserving textual token order and visual spatial structure. The backbone learns a single vector field over these hyperchunks through Flow Matching. Joint attention enables cross-modal interaction, while modality-specific feed-forward networks process each modality. The model predicts multiple target chunks in parallel during training and generates hyperchunks sequentially at inference. We instantiate MF-1 and pretrain it on multimodal data. Across 0.6B, 1.2B, and 1.6B scales, continued pretraining consistently improves multimodal modeling. With only 150B pretraining tokens, MF-1 achieves an average score of 82.8 across GenEval and DPG-Bench and 75.3 across VQAv2, MMBench, and POPE, remaining competitive with unified models trained on substantially more data. Under matched data, optimization, and parameter budgets, Multimodal Flow further outperforms representative hybrid and discrete models. These results establish continuous chunk-based embedding flow modeling as a new fully continuous paradigm for unified multimodal modeling. The related code and model are publicly released at https://github.com/hustvl/Multimodal-Flow.
