PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning

2026-08-17

AI Summary

This study addresses the unreliability of long-horizon planning in large language models and the absence of automated verification benchmarks for PDDL generation. We propose an iterative agent-based PDDL generation framework and introduce NL-PDDLGym, a novel benchmark supporting executable environment validation. This approach leverages agent feedback mechanisms to ensure reliable natural-language-to-symbolic planning translation with automated verification. Experimental results demonstrate that our method achieves an 89.6% valid plan generation rate on the test set, significantly outperforming existing PDDL generation approaches and direct LLM planning baselines. These findings confirm that integrating iterative feedback with executable verification effectively enhances both the accuracy and robustness of symbolic planning derived from natural language specifications.

Abstract

LLMs remain unreliable for long-horizon planning, often generating logically inconsistent or non-applicable plans. Recent hybrid methods instead translate natural language into the Planning Domain Definition Language (PDDL), allowing symbolic planners to produce verifiable plans. However, existing methods frequently rely on rigid generation pipelines, a partial PDDL definition, or human feedback. Furthermore, their evaluation is hindered by the lack of standardized benchmarks with automated verification. To address these limitations, we present PDDLCoder, an agentic framework for PDDL generation from natural language that iteratively generates, analyzes, and refines planning specifications. We further introduce NL-pddlgym, a benchmark dataset comprising 711 planning problems across 23 domains with executable gym environments for the automated verification of plan applicability. Experiments on the NL-pddlgym test set containing 106 problems across 4 held-out domains show that PDDLCoder generates applicable plans for 89.6\% of tested planning problems. This improves upon our adaptations of previous PDDL generation methods, which achieved up to 45.3\%, and outperforms direct LLM planning approaches, which reached up to 74.5\% on the same test set. Our work demonstrates the effectiveness of agentic PDDL generation for planning and establishes a reproducible benchmark for future research on LLM-assisted symbolic planning.

Problem

• LLM-assisted symbolic planning
• PDDL generation
• long-horizon planning
• automated verification
• standardized benchmarks

Innovation

• Agentic PDDL Generation
• Iterative Refinement
• NL-pddlgym Benchmark
• Symbolic Planning
• Automated Verification