ReasonPlanner: Enhancing Autonomous Planning in Dynamic Environments with Temporal Knowledge Graphs and LLMs

2024-10-11

arXiv.org

AI Summary

Scientific laboratories demand autonomous agents capable of complex, dynamic, safety-critical reasoning with strong temporal understanding and interpretability. Method: We propose a world model framework grounded in Temporal Knowledge Graphs (TKGs), uniquely coupling a frozen large language model (LLM) with an explicit TKG to enable training-free, interpretable planning; we further introduce a natural-language-driven actor-critic mechanism for online replanning and zero-shot deployment. Contributions/Results: Key innovations include (i) TKG-based modeling of environmental dynamics, (ii) frozen LLMs ensuring stability and auditability, and (iii) reflective trajectory planning for robust decision-making. Evaluated on ScienceWorld, our approach achieves 1.8× the performance of state-of-the-art prompting methods, with significantly improved sample efficiency, no gradient updates, and an out-of-the-box interactive interface designed for non-ML domain scientists.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Modeling other Agents

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Planning and performing interactive tasks, such as conducting experiments to determine the melting point of an unknown substance, is straightforward for humans but poses significant challenges for autonomous agents. We introduce ReasonPlanner, a novel generalist agent designed for reflective thinking, planning, and interactive reasoning. This agent leverages LLMs to plan hypothetical trajectories by building a World Model based on a Temporal Knowledge Graph. The agent interacts with the environment using a natural language actor-critic module, where the actor translates the imagined trajectory into a sequence of actionable steps, and the critic determines if replanning is necessary. ReasonPlanner significantly outperforms previous state-of-the-art prompting-based methods on the ScienceWorld benchmark by more than 1.8 times, while being more sample-efficient and interpretable. It relies solely on frozen weights thus requiring no gradient updates. ReasonPlanner can be deployed and utilized without specialized knowledge of Machine Learning, making it accessible to a wide range of users.

Problem

• Designing a generalist scientific agent for laboratory tasks
• Addressing delicate scientific tasks requiring advanced reasoning
• Overcoming limitations in structured temporal environment understanding

Innovation

• Structured and temporal memory for planning
• Multi-turn retrieval system for reasoning
• Interactive retrieval method in RAG pipeline