LTLCrit: A Temporal Logic-based LLM Critic for Safe and Efficient Embodied Agents

2025-07-04

AI Summary

Large language models (LLMs) suffer from error accumulation, insufficient safety guarantees, and suboptimal efficiency in long-horizon embodied planning. Method: We propose a modular actor-critic architecture integrating natural language reasoning with formal logic. The core innovation is LTLCrit—a critic module that automatically synthesizes Linear Temporal Logic (LTL) constraints to uniformly encode both static safety rules and dynamically learned soft constraints. An LLM serves as the actor, generating actions grounded in natural language observations and symbolic graph traversal; LTLCrit monitors execution traces in real time and provides corrective LTL feedback, enabling logic-guided self-supervision. Results: Evaluated on a Minecraft mining task, our approach achieves 100% task completion—significantly outperforming existing baselines—and demonstrates that synergistic integration of LLMs and formal logic enhances robustness and reliability in long-term planning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) have demonstrated promise in reasoning tasks and general decision-making in static environments. In long-term planning tasks, however, errors tend to accumulate, often leading to unsafe or inefficient behavior, limiting their use in general-purpose settings. We propose a modular actor-critic architecture in which an LLM actor is guided by LTLCrit, a trajectory-level LLM critic that communicates via linear temporal logic (LTL). Our setup combines the reasoning strengths of language models with the guarantees of formal logic. The actor selects high-level actions from natural language observations, while the critic analyzes full trajectories and proposes new LTL constraints that shield the actor from future unsafe or inefficient behavior. The architecture supports both fixed, hand-specified safety constraints and adaptive, learned soft constraints that promote long-term efficiency. Our architecture is model-agnostic: any LLM-based planner can serve as the actor, and LTLCrit serves as a logic-generating wrapper. We formalize planning as graph traversal under symbolic constraints, allowing LTLCrit to analyze failed or suboptimal trajectories and generate new temporal logic rules that improve future behavior. We evaluate our system on the Minecraft diamond-mining benchmark, achieving 100% completion rates and improving efficiency compared to baseline LLM planners. Our results suggest that enabling LLMs to supervise each other through logic is a powerful and flexible paradigm for safe, generalizable decision making.

Problem

• Prevent unsafe behavior in LLM-based embodied agents
• Reduce error accumulation in long-term planning tasks
• Combine LLM reasoning with formal logic guarantees

Innovation

• Modular actor-critic with LTL-based critic
• Combines LLM reasoning with formal logic
• Generates adaptive LTL constraints for safety