Search papers, labs, and topics across Lattice.
This paper introduces a self-evolving training framework for unified large multimodal models (LMMs) that autonomously enhances visual understanding and image generation using only unlabeled images. By employing a Proposer, Solver, and Generator structure, the method leverages self-derived consistency signals for training, eliminating the need for human annotations or external reward models. The approach demonstrates significant improvements across eight understanding metrics, achieving a 3.5% absolute gain on the MMMU benchmark and enhancing image generation performance from 82% to 85% on the BAGEL model.
Unsupervised training can elevate multimodal models, achieving a 3.5% boost in understanding metrics and a notable increase in image generation fidelity without human intervention.
Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a $+3.5\%$ absolute gain on MMMU and improves GenEval image generation performance from $82\%$ to $85\%$. Code and models are publicly released.
