NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions

2024-05-07

arXiv.org

AI Summary

Traditional PDDL planning requires expert domain modeling, while purely LLM-based approaches lack reliability and formal verifiability. Method: We propose the first domain-agnostic, offline LLM-driven planning framework that generates executable, verifiable PDDL domain and problem files end-to-end from minimal natural language descriptions—fully autonomously, without human intervention. Our approach integrates incremental semantic parsing with a dual-PDDL generation mechanism and introduces an interpretable intermediate representation to ensure plan inspectability and modifiability. The system combines a lightweight LLM, classical planners (FF/Fast Downward), and customized prompting strategies. Results: Evaluated on 15 tasks across four standard planning domains, our framework successfully solves 10 tasks—substantially outperforming a chain-of-thought LLM baseline (2 solved). Crucially, it proactively flags failure cases rather than producing invalid plans, enhancing trustworthiness and debuggability.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Today's classical planners are powerful, but modeling input tasks in formats such as PDDL is tedious and error-prone. In contrast, planning with Large Language Models (LLMs) allows for almost any input text, but offers no guarantees on plan quality or even soundness. In an attempt to merge the best of these two approaches, some work has begun to use LLMs to automate parts of the PDDL creation process. However, these methods still require various degrees of expert input. We present NL2Plan, the first domain-agnostic offline LLM-driven planning system. NL2Plan uses an LLM to incrementally extract the necessary information from a short text prompt before creating a complete PDDL description of both the domain and the problem, which is finally solved by a classical planner. We evaluate NL2Plan on four planning domains and find that it solves 10 out of 15 tasks - a clear improvement over a plain chain-of-thought reasoning LLM approach, which only solves 2 tasks. Moreover, in two out of the five failure cases, instead of returning an invalid plan, NL2Plan reports that it failed to solve the task. In addition to using NL2Plan in end-to-end mode, users can inspect and correct all of its intermediate results, such as the PDDL representation, increasing explainability and making it an assistive tool for PDDL creation.

Problem

• Automating PDDL task generation from minimal natural language descriptions
• Eliminating expert input requirements for classical planning systems
• Ensuring plan quality guarantees while using natural language inputs

Innovation

• Automatically generates complete PDDL tasks
• Uses LLM to incrementally extract information
• Solves tasks with classical planners for guarantees