Search papers, labs, and topics across Lattice.
This paper investigates the capability of contemporary Large Language Models (LLMs) to automatically translate unstructured natural language requirements into Linear Temporal Logic (LTL) specifications, a critical task in formal methods for safety-critical systems. By employing a few-shot prompting strategy across six LLMs on a diverse benchmark of 15 requirements, the authors generated 450 candidate LTL formulas and evaluated their performance through semantic assessments and reproducibility measures. The results reveal that these LLMs can produce meaningful LTL outputs without the need for task-specific fine-tuning, indicating their potential as effective tools in formalization workflows.
Current LLMs can generate precise LTL specifications from unstructured requirements, achieving significant performance without fine-tuning.
Automatically translating unstructured natural language requirements into formal specifications remains a challenge in requirements engineering and formal methods, particularly for safety- and mission-critical systems whose verification depends on mathematically precise specifications. This paper evaluates whether contemporary off-the-shelf Large Language Models (LLMs) can help bridge this gap by generating Linear Temporal Logic (LTL) formulas directly from unstructured requirements. The study examines six modern LLMs using a few-shot prompting strategy on a heterogeneous benchmark of 15 structurally varied requirements. Five independent generations were collected for each requirement-model pair, yielding 450 candidate LTL formulas in total. Performance was assessed through manual semantic evaluation, pass@k for k in {1, 3, 5}, and a self-consistency measure capturing syntactic reproducibility across stochastic trials. The results indicate that current general-purpose LLMs can achieve practically significant performance on the unstructured NL-to-LTL task without task-specific fine-tuning. The study also considers understandability for non-experts by pairing generated formulas with model-produced natural language explanations and discussing the complementary use of timeline-based LTL visualization. The findings suggest that modern LLMs are becoming viable front-end assistants for semi-automated formalization workflows.
