Search papers, labs, and topics across Lattice.
This paper introduces ILLUME-X, a unified multimodal model designed for high-quality, free-form interleaved text-image generation, addressing the challenges of multimodal data efficiency and training stability. The model employs an expanded training data pipeline, a progressive training strategy with self-adaptive objectives, and a novel evaluation method called ILScore. ILLUME-X demonstrates superior performance over existing models in various tasks, including style transfer, image decomposition, and storytelling, highlighting its effectiveness in multimodal AI applications.
ILLUME-X achieves unprecedented quality in free-form interleaved text-image generation, setting a new benchmark for multimodal models.
The advancement of generative AI models capable of producing text and image marks a critical step forward in the realm of multimodal intelligence, particularly for tasks involving the interleaving of both modalities. To advance this intelligence to the next stage, it is crucial for models to autonomously generate free-form interleaved text-image sequences. In this paper, we introduce ILLUME-X, an advanced unified multimodal paradigm that enables high-quality, free-form interleaved text-image generation by improving multimodal data efficiency and stabilizing the multimodal training process. ILLUME-X comprises three key components: (i) an expanded training data pipeline optimized for interleaved text-image generation, (ii) a progressive training strategy with self-adaptive objectives for free-length multimodal token sequences, and (iii) an objective and comprehensive evaluation method ILScore for interleaved text-image sequences. Notably, our ILLUME-X outperforms previous unified models across multiple interleaved text-image generation tasks like style transfer, image decomposition and storytelling.
