Search papers, labs, and topics across Lattice.
This paper introduces PDDLCoder, an agentic framework that enhances the generation of Planning Domain Definition Language (PDDL) from natural language by iteratively generating, analyzing, and refining planning specifications. The authors also present NL-pddlgym, a comprehensive benchmark dataset that includes 711 planning problems across 23 domains, facilitating automated verification of plan applicability. Experimental results show that PDDLCoder achieves an impressive 89.6% applicability rate on the test set, significantly outperforming previous methods and direct LLM planning approaches, which had maximum applicability rates of 45.3% and 74.5%, respectively.
PDDLCoder achieves nearly 90% applicability in long-horizon planning, setting a new standard for LLM-assisted symbolic planning.
LLMs remain unreliable for long-horizon planning, often generating logically inconsistent or non-applicable plans. Recent hybrid methods instead translate natural language into the Planning Domain Definition Language (PDDL), allowing symbolic planners to produce verifiable plans. However, existing methods frequently rely on rigid generation pipelines, a partial PDDL definition, or human feedback. Furthermore, their evaluation is hindered by the lack of standardized benchmarks with automated verification. To address these limitations, we present PDDLCoder, an agentic framework for PDDL generation from natural language that iteratively generates, analyzes, and refines planning specifications. We further introduce NL-pddlgym, a benchmark dataset comprising 711 planning problems across 23 domains with executable gym environments for the automated verification of plan applicability. Experiments on the NL-pddlgym test set containing 106 problems across 4 held-out domains show that PDDLCoder generates applicable plans for 89.6\% of tested planning problems. This improves upon our adaptations of previous PDDL generation methods, which achieved up to 45.3\%, and outperforms direct LLM planning approaches, which reached up to 74.5\% on the same test set. Our work demonstrates the effectiveness of agentic PDDL generation for planning and establishes a reproducible benchmark for future research on LLM-assisted symbolic planning.
