Search papers, labs, and topics across Lattice.
EchoGen is a unified framework for layout-to-image generation and image grounding, leveraging the strengths of each task to improve the other. The method employs a progressive training strategy involving Parallel Multi-Task Pre-training (PMTP), Dual Joint Optimization (DJO), and Cycle RL with a GRPO strategy to eliminate reliance on visual supervision. Experiments demonstrate state-of-the-art results on both layout-to-image generation and image grounding benchmarks, showcasing synergistic gains from the joint optimization.
By unifying layout-to-image generation and image grounding with a novel cycle-consistent learning approach, EchoGen achieves state-of-the-art results in both tasks, proving that solving two problems at once can be better than solving them separately.
In this work, we present EchoGen, a unified framework for layout-to-image generation and image grounding, capable of generating images with accurate layouts and high fidelity to text descriptions (e.g., spatial relationships), while grounding the image robustly at the same time. We believe that image grounding possesses strong text and layout understanding abilities, which can compensate for the corresponding limitations in layout-to-image generation. At the same time, images generated from layouts exhibit high diversity in content, thereby enhancing the robustness of image grounding. Jointly training both tasks within a unified model can promote performance improvements for each. However, we identify that this joint training paradigm encounters several optimization challenges and results in restricted performance. To address these issues, we propose progressive training strategies. First, the Parallel Multi-Task Pre-training (PMTP) stage equips the model with basic abilities for both tasks, leveraging shared tokens to accelerate training. Next, the Dual Joint Optimization (DJO) stage exploits task duality to sequentially integrate the two tasks, enabling unified optimization. Finally, the Cycle RL stage eliminates reliance on visual supervision by using consistency constraints as rewards, significantly enhancing the model's unified capabilities via the GRPO strategy. Extensive experiments demonstrate state-of-the-art results on both layout-to-image generation and image grounding benchmarks, and reveal clear synergistic gains from optimizing the two tasks together.
