Search papers, labs, and topics across Lattice.
This paper introduces LongWebBench, a comprehensive benchmark designed to evaluate the structural and functional generation of long webpages by vision-language models (VLMs). The benchmark comprises 490 real-world long webpages for assessing structural fidelity and 507 interaction tasks for functional evaluation, employing both a VLM-based metric and a DOM-augmented agent-based pipeline. Results indicate that while structural fidelity diminishes with increased webpage length, visually plausible outputs often fail to facilitate multi-step interactions, underscoring the necessity for evaluations that prioritize executable functionality over mere visual similarity.
Structural fidelity in webpage generation drops significantly with length, revealing that visual appeal doesn't guarantee functional effectiveness in VLMs.
Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages. We introduce LongWebBench, a benchmark for evaluating long-horizon webpage generation from both structural and functional perspectives. LongWebBench contains 490 real-world long webpages for structural fidelity evaluation and 507 goal-oriented interaction tasks over 129 webpages for functional evaluation. It employs two complementary protocols: a multi-dimensional VLM-based metric for assessing long-range structural coherence, and a DOM-augmented agent-based pipeline for end-to-end functional verification. We further examine the automatic evaluation protocols through human agreement analysis. Experiments with state-of-the-art open-source and proprietary VLMs under single-image and multi-image settings reveal that structural fidelity degrades as webpage length increases, while visually plausible generations often fail to support executable multi-step interactions. These results highlight the need to evaluate long webpage generation beyond visual similarity, with executable interaction as a core criterion. Our code and data are available at https://github.com/zheny2751-dotcom/LongWebBench.
Tsinghua AI