Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper tackles the challenges of transforming natural language (NL) requirements into graphical user interfaces (GUIs) and verifying those requirements, addressing the ambiguity inherent in NL. By introducing novel NL-based GUI retrieval and reranking methods along with LLM adaptations for GUI generation, the authors demonstrate significant reductions in the manual effort required for prototyping. Additionally, they present LLM-based verification techniques that effectively assess both functional and non-functional requirements in dynamic GUI applications, showcasing their methods on a comprehensive benchmark with human annotations.
Transforming natural language requirements into high-fidelity GUI prototypes can be achieved with unprecedented efficiency using advanced LLM techniques.
Requirements elicitation is essential for developing interactive software systems, as it helps ensure that the resulting product meets stakeholder needs. Since elicitation typically relies on natural language (NL), misunderstandings can arise from its inherent ambiguity. Formal specifications can reduce ambiguity but require technical expertise. GUI prototyping therefore provides a valuable alternative by turning requirements into tangible visual artifacts that support communication, elicitation, and validation. However, creating high-fidelity prototypes remains time-consuming and costly. Similarly, requirements verification, which ensures that implementations conform to specified requirements, is still largely manual, while existing automated approaches are often limited to static, rule-based techniques. This work addresses two challenges: (C1) reducing the effort required to transform NL requirements into GUI prototypes, and (C2) reducing the effort required for requirements verification in GUI applications and prototypes. For C1, we introduce novel NL-based GUI retrieval and reranking methods, new benchmarks, and techniques for efficiently adapting LLMs to GUI generation, including proprietary GUI representations. Their effectiveness is demonstrated on a large benchmark with human annotations. For C2, we propose LLM-based methods for verifying semantically complex NL requirements on static GUI prototypes and introduce a multimodal LLM-based agent for verifying complex functional and non-functional requirements in dynamic GUI applications through automatically generated and evaluated interaction trajectories. Overall, the proposed methods substantially reduce manual effort in GUI prototyping and requirements verification.