Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
This paper introduces MM-WebAgent, a hierarchical agentic framework that leverages AIGC tools for multimodal webpage generation by coordinating element creation through hierarchical planning and iterative self-reflection. The framework addresses style inconsistency and poor global coherence issues common in direct AIGC integration by jointly optimizing layout, multimodal content, and their integration. Experiments show MM-WebAgent surpasses code-generation and agent-based baselines, particularly in multimodal element generation and integration, as validated by a newly introduced benchmark and multi-level evaluation protocol.
Hierarchical planning and self-reflection can finally wrangle AIGC tools into producing coherent, visually consistent webpages.
The rapid progress of Artificial Intelligence Generated Content (AIGC) tools enables images, videos, and visualizations to be created on demand for webpage design, offering a flexible and increasingly adopted paradigm for modern UI/UX. However, directly integrating such tools into automated webpage generation often leads to style inconsistency and poor global coherence, as elements are generated in isolation. We propose MM-WebAgent, a hierarchical agentic framework for multimodal webpage generation that coordinates AIGC-based element generation through hierarchical planning and iterative self-reflection. MM-WebAgent jointly optimizes global layout, local multimodal content, and their integration, producing coherent and visually consistent webpages. We further introduce a benchmark for multimodal webpage generation and a multi-level evaluation protocol for systematic assessment. Experiments demonstrate that MM-WebAgent outperforms code-generation and agent-based baselines, especially on multimodal element generation and integration. Code&Data: https://aka.ms/mm-webagent.
Microsoft Research