CoT-TL: Low-Resource Temporal Knowledge Representation of Planning Instructions Using Chain-of-Thought Reasoning

2024-10-14

IEEE/RJS International Conference on Intelligent RObots and Systems

AI Summary

Autonomous agents struggle to accurately translate ambiguous natural-language planning instructions into Linear Temporal Logic (LTL) under low-resource conditions. Method: We propose a fine-tuning-free, data-efficient context-learning framework that integrates chain-of-thought reasoning with semantic role labeling to jointly guide LTL generation; introduces real-time syntactic validation via model checking to suppress hallucination; and combines zero-shot in-context learning, LTL grammar-constrained decoding, and formal verification. Contribution/Results: Our approach achieves state-of-the-art accuracy across three low-data benchmarks, generalizes to unseen LTL structures, and enhances interpretability and user trust. It has been successfully deployed on a quadcopter for real-time, multi-step natural-language-driven task planning—demonstrating robustness and practical applicability in resource-constrained settings.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Multiagent Systems: Multiagent Planning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Autonomous agents often face the challenge of interpreting uncertain natural language instructions for planning tasks. Representing these instructions as Linear Temporal Logic (LTL) enables planners to synthesize actionable plans. We introduce CoT-TL, a data-efficient in-context learning framework for translating natural language specifications into LTL representations. CoT-TL addresses the limitations of large language models, which typically rely on extensive fine-tuning data, by extending chain-of-thought reasoning and semantic roles to align with the requirements of formal logic creation. This approach enhances the transparency and rationale behind LTL generation, fostering user trust. CoT-TL achieves state-of-the-art accuracy across three diverse datasets in low-data scenarios, outperforming existing methods without fine-tuning or intermediate translations. To improve reliability and minimize hallucinations, we incorporate model checking to validate the syntax of the generated LTL output. We further demonstrate CoT-TL’s effectiveness through ablation studies and evaluations on unseen LTL structures and formulas in a new dataset. Finally, we validate CoT-TL’s practicality by integrating it into a QuadCopter for multi-step drone planning based on natural language instructions.

Problem

• Translating natural language instructions into Linear Temporal Logic representations
• Addressing data scarcity in formal logic generation for planning tasks
• Enhancing reliability of language-to-logic translation with model checking

Innovation

• Uses chain-of-thought reasoning for LTL generation
• Incorporates model checking to validate syntax
• Achieves high accuracy without fine-tuning data