Search papers, labs, and topics across Lattice.
This paper introduces the Evidence-Grounded, Social-Weighted Persona Panel (ESPP), a novel three-stage evaluation method for Generative UI that leverages a diverse panel of personas to assess interface quality. By employing a semantically-gated bounded-confidence mechanism and aggregating ratings through a Delphi-inspired approach, ESPP significantly improves the correlation with human judgment, achieving a Pearson correlation coefficient increase from 0.716 to 0.922. Additionally, the method reveals critical insights into how different user subgroups perceive interfaces, highlighting structural disagreements that a single judge would overlook.
ESPP not only enhances the fidelity of GenUI evaluations but also uncovers nuanced user group divergences that traditional methods miss.
Generative UI (GenUI) lets large language models synthesize a complete, renderable interface directly from a natural-language instruction, but evaluating the quality of what they generate remains an open problem. Human evaluation is costly and rater-variant, while LLM-as-a-judge is scalable but reflects only a single implicit viewpoint, unable to capture how different populations of real users actually perceive the same interface. We propose the Evidence-Grounded, Social-Weighted Persona Panel (ESPP), a three-stage GenUI evaluation method in which a panel of psychologically diverse, evidence-grounded personas independently rates a screenshot, exchanges opinions under a trait-derived, semantically-gated bounded-confidence mechanism, and is aggregated via Delphi-inspired social weighting into a single judgment. ESPP tracks human judgment substantially more closely than a naive single-pass judge, raising Pearson $r$ from $0.716$ to $0.922$, and a prompt-ensemble control recovers only about a third of this gap, isolating genuine persona and evidence grounding as the dominant source of improvement. Beyond this fidelity gain, retaining each panelist's individual rating further reveals that user subgroups agree on overall model rankings yet diverge sharply on specific rating dimensions, a structural disagreement a single homogeneous judge would systematically erase. The codes are available at https://github.com/Wuzheng02/ESPP.
