Search papers, labs, and topics across Lattice.
The paper introduces Cookie-Bench, a novel evaluation framework for web generation that addresses the limitations of existing methods by being reference-free, autonomously driven, and holistically reasoned. Cookie-Bench comprises a new WebDev benchmark with diverse tasks and difficulty levels, along with a metacognitive evaluation framework that separates evidence accumulation from judgment. Experiments across 13 frontier LLMs demonstrate that the framework aligns with human expert ratings and reveals significant potential for improvement in interactive web generation.
Current LLMs have substantial headroom for improvement in interactive web generation, a fact revealed by a new benchmark that mirrors human expert ratings without relying on reference implementations.
Front-end web code has become a core product surface for every frontier LLM release, yet evaluating these interactive applications at development speed remains costly because human-judged leaderboards like Arena do not scale. Existing automated proxies typically lean on reference implementations, test suites, or rigid checklists, and tend to miss the reasoned synthesis a human reviewer performs over a live session. We articulate a new evaluation regime that is simultaneously reference-free, autonomously driven, and holistically reasoned, and instantiate it through two artifacts. \textbf{\dataname} is an 11-domain, 54-leaf, 1,000-query WebDev benchmark spanning both static-presentation and interactive-application tasks, balanced across three difficulty tiers and three target-language groups, with briefs rewritten to resist recall from circulated prompts. \textbf{\framename}, grounded in Flavell's metacognitive monitoring, separates evidence accumulation from judgment across three stages: Static Perception forms a first impression from passive observation; Agent-Driven Interaction explores the application autonomously while capturing continuous screen video, audio, and per-step screenshots; Dynamic Scoring issues holistic functionality and aesthetics verdicts with structured failure attribution only after the evidence chain is complete. On \dataname, \framename aligns closely with expert human ratings while surfacing substantial headroom across 13 frontier LLMs on interactive web generation. \noindenthttps://anonymous.4open.science/r/Cookie-3CE/
