Agent Planning with World Knowledge Model

2024-05-23

arXiv.org

AI Summary

To address the global盲目 exploration and local action hallucination of LLM-based agents in complex interactive planning, this paper proposes a Parameterized World Knowledge Model (WKM). The WKM integrates expert knowledge with prior task knowledge and dynamic state knowledge—both derived from expert demonstrations and self-sampled trajectories—enabling a learnable dual-track planning mechanism: prior knowledge guides global strategy selection, while dynamic knowledge constrains local action generation. Built upon open-source LLMs (e.g., Mistral-7B, Gemma-7B, Llama-3-8B), our method leverages trajectory distillation and knowledge self-synthesis. Evaluated on three real-world simulation benchmarks, it significantly outperforms strong baselines, demonstrating (i) instance-level knowledge transfer across tasks, (ii) the feasibility of guiding powerful LLMs with lightweight WKMs, and (iii) the potential for unified WKM training. The approach effectively mitigates trial-and-error inefficiency and action hallucination. Code is publicly available.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent endeavors towards directly using large language models (LLMs) as agent models to execute interactive planning tasks have shown commendable results. Despite their achievements, however, they still struggle with brainless trial-and-error in global planning and generating hallucinatory actions in local planning due to their poor understanding of the ``real'' physical world. Imitating humans' mental world knowledge model which provides global prior knowledge before the task and maintains local dynamic knowledge during the task, in this paper, we introduce parametric World Knowledge Model (WKM) to facilitate agent planning. Concretely, we steer the agent model to self-synthesize knowledge from both expert and sampled trajectories. Then we develop WKM, providing prior task knowledge to guide the global planning and dynamic state knowledge to assist the local planning. Experimental results on three complex real-world simulated datasets with three state-of-the-art open-source LLMs, Mistral-7B, Gemma-7B, and Llama-3-8B, demonstrate that our method can achieve superior performance compared to various strong baselines. Besides, we analyze to illustrate that our WKM can effectively alleviate the blind trial-and-error and hallucinatory action issues, providing strong support for the agent's understanding of the world. Other interesting findings include: 1) our instance-level task knowledge can generalize better to unseen tasks, 2) weak WKM can guide strong agent model planning, and 3) unified WKM training has promising potential for further development. The code is available at https://github.com/zjunlp/WKM.

Problem

• Large Language Models
• Planning Tasks
• Real-world Understanding

Innovation

• World Knowledge Model
• Self-learning Expert Knowledge
• Enhanced Agent Performance