Search papers, labs, and topics across Lattice.
This paper investigates the optimization of generation order in Diffusion Language Models (DLMs) for text-to-image synthesis and multimodal understanding, revealing that traditional methods relying solely on model logits are inadequate for determining optimal sequences. By introducing a learnable control module trained via Group Relative Policy Optimization (GRPO), the authors significantly enhance the model's performance in capturing spatial relationships and improving multimodal reasoning. The proposed method achieves notable improvements of 4.08% in text-to-image alignment and 4.85% in multimodal understanding, underscoring its effectiveness across applications.
Learning the generation order in multimodal tasks can boost performance by over 4%鈥攁 game changer for DLMs.
Diffusion Language Models (DLMs) have recently achieved substantial progress in natural language generation tasks. Recent research demonstrates that adaptive token generation ordering can significantly improve performance in mathematical reasoning and code synthesis applications. In this work, we investigate the optimization of generation order for both text-to-image synthesis and multimodal understanding. We first establish that, unlike structured problems in language generation such as Sudoku puzzles, model logits alone are insufficient for determining optimal generation sequences in text-to-image generation and multimodal understanding. To address this challenge, we introduce a learnable control module trained via Group Relative Policy Optimization (GRPO) to determine the generation order. Our results demonstrate that learning this control block substantially improves both text-to-image alignment and multimodal understanding in DLMs. In particular, it enhances the model's ability to capture fine-grained spatial relationships in generated images while also strengthening performance on multimodal reasoning and comprehension tasks. We evaluate our framework on GenEval, an object-focused benchmark for text-to-image alignment, where it achieves 4.08% relative improvements. In addition, experiments on VLMEvalKit confirm 4.85% relative improvements in multimodal understanding, highlighting the broad effectiveness of our approach.
