Training the Orchestrator: A Supervised Approach to End-to-End PDDL Planning with LLM Agents

2026-06-19

AI Summary

This work addresses the semantic gap between natural language planning instructions and the formal input required by classical planners by introducing HALO, a framework that automatically translates human directives into verifiable PDDL plans. HALO leverages the complete sequence of correct decisions provided by a planner validator as a strong supervision signal to train a lightweight orchestrator, augmented with three hard-coded rules for handling simple cases. The framework employs a small policy model fine-tuned via QLoRA, a multi-agent collaborative architecture, and a validation-feedback-driven supervised learning paradigm, thereby avoiding per-step large language model (LLM) invocations or reliance on sparse rewards. Experiments demonstrate that HALO matches or exceeds the performance of GPT-5-mini prompting baselines and approaches that of Gemini-3-Flash across multiple planning benchmarks, while reducing orchestration costs by over an order of magnitude—approximately 1/45th of GPT-5-mini and 1/15th of Gemini-3-Flash—and decreasing LLM calls per task by 40–50%.

Abstract

Translating natural-language planning intent into verified plans is a longstanding challenge: people communicate goals in language, while classical planners require formal PDDL specifications. Recent agentic frameworks bridge this gap by orchestrating a pool of specialized repair agents inside a verifier-checked refinement loop, but the orchestrator at the centre is itself a prompted frontier LLM, paying a frontier-LLM API call at every refinement step. We present HALO (Hybrid Agent-Learned Orchestrator), which trains the orchestrator from refinement trajectories that an external verifier has certified as ending in valid plans, across 11 PDDL domains. HALO pairs a small QLoRA-tuned policy with three hardcoded rules for trivially decidable selections, and operates over an expanded 21-agent action space. Unlike approaches that prompt a frontier LLM at every step or learn an orchestrator from sparse end-of-episode rewards, our key observation is that the verifier already provides strong guidance: every accepted trajectory is a sequence of demonstrably correct (state, agent) decisions, directly usable as supervision. Across PlanBench, Natural Plan, and classical planning benchmarks, HALO matches or exceeds the GPT-5-mini prompted baseline on success rate, sits within three percentage points of the stronger Gemini-3-Flash prompted baseline, reduces orchestration cost by more than an order of magnitude (\$0.18 to \$0.004 per task against GPT-5-mini, roughly 45$\times$ cheaper; roughly 15$\times$ cheaper than Gemini-3-Flash), and cuts total LLM calls per episode by 40 to 50 percent.

Problem

• PDDL planning
• natural language to formal specification
• LLM-based planning
• plan verification
• orchestration cost

Innovation

• supervised orchestration
• PDDL planning
• LLM agents
• trajectory supervision
• cost-efficient planning