Addressing the Challenges of Planning Language Generation

2025-05-20

AI Summary

This work investigates the feasibility of generating formal PDDL planning descriptions using open-source small-parameter language models (<50B)—a task previously deemed beyond their capability. To address the limited effectiveness of conventional grammar-constrained decoding, we propose a solver-feedback-driven iterative refinement mechanism at inference time: satisfiability feedback from a symbolic planner serves as a correction signal, integrated with a plan validator to enable closed-loop optimization. Our lightweight PDDL generation pipeline achieves substantial improvements across multiple benchmarks, more than doubling success rates over strong baselines. The core contribution is the first systematic demonstration that small-scale open-source LLMs can reliably support symbolic planning generation through inference-time augmentation—without model fine-tuning or scaling—thereby establishing a novel paradigm for synergistic integration of LLMs with classical AI planning systems.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Using LLMs to generate formal planning languages such as PDDL that invokes symbolic solvers to deterministically derive plans has been shown to outperform generating plans directly. While this success has been limited to closed-sourced models or particular LLM pipelines, we design and evaluate 8 different PDDL generation pipelines with open-source models under 50 billion parameters previously shown to be incapable of this task. We find that intuitive approaches such as using a high-resource language wrapper or constrained decoding with grammar decrease performance, yet inference-time scaling approaches such as revision with feedback from the solver and plan validator more than double the performance.

Problem

• Generating formal planning languages using open-source LLMs
• Evaluating 8 PDDL generation pipelines with sub-50B parameter models
• Improving performance via solver feedback and plan validation

Innovation

• Open-source LLMs generate PDDL effectively
• Solver feedback doubles planning performance
• Constrained decoding reduces generation quality