Metraipolis › archive › dc_d6bdb4724bf7
Grounded Evaluation and Repair for NL-to-PDDL Problem Generation — arXiv:2609.09898
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.09898
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T07:24:35.292Z
- SHA-256 of the archived text
25a0cc36929e7f2035662d0ef94d28e5b4d535a2a433f735298b06ee44c141fd- Stable URL of this page
https://metraipolis.com/documento/dc_d6bdb4724bf7
How to cite
Grounded Evaluation and Repair for NL-to-PDDL Problem Generation — arXiv:2609.09898 — arXiv (en), https://arxiv.org/abs/2609.09898 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d6bdb4724bf7 (Metraipolis archive dc_d6bdb4724bf7, sha256 25a0cc36929e7f2035662d0ef94d28e5b4d535a2a433f735298b06ee44c141fd)
BibTeX
@misc{dcd6bdb4724bf7,
title = {Grounded Evaluation and Repair for NL-to-PDDL Problem Generation — arXiv:2609.09898},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.09898},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d6bdb4724bf7; sha256 25a0cc36929e7f2035662d0ef94d28e5b4d535a2a433f735298b06ee44c141fd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09898. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) have shown promise for translating Natural Language (NL) planning descriptions into PDDL problem instances. However, standard evaluation criteria such as syntactic validity or planner success can substantially overestimate faithfulness to the described task: a generated problem may be parseable and solvable while misrepresenting the intended initial state, goal, object structure, or optimization target. This paper studies an end-to-end NL-to-PDDL pipeline that combines LLM generation, checks in terms of PDDL parsing, planning and validation, a domain-conformance checker, an LLM critic, and iterative repair. Fine-grained repair feedback is constructed from the domain description, the generated problem, the natural language problem description, and operational diagnostics. Reference-based comparisons against curated benchmark PDDL problem descriptions are used for post-hoc benchmark analysis, and these offline checks include renaming-invariant structural matching and semantic equivalence, where domain support is available. Across Planetarium, AutoPlanBench, and curated PDDL~2.1 problems, results show that operational success and benchmark-reference reconstruction can diverge substantially. Results also show that structured repair can be useful, and that PDDL~2.1 remains challenging for reference reconstruction, even when operational success improves. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.09898 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T07:00:16.228Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).