Self-Evolving World Models for LLM Agent Planning

2026-06-29

AI Summary

This work addresses the challenge that large language model (LLM) agents often suffer from unreliable predictions in long-horizon planning, which degrades decision-making performance. To mitigate this issue without updating model parameters, the authors propose WorldEvolver—a test-time context self-evolution framework that enhances the reliability of world models and planning efficacy. WorldEvolver integrates three core components: episodic memory, semantic memory, and selective prediction, leveraging retrieval-based simulation, prediction-observation discrepancy heuristics, and low-confidence prediction filtering to dynamically refine memory representations and improve prediction fidelity. Experiments demonstrate that WorldEvolver significantly boosts both prediction accuracy and task success rates across three prominent backbone LLMs on the ALFWorld and ScienceWorld benchmarks, consistently outperforming existing world model baselines.

Abstract

World models offer a principled way to equip long-horizon LLM agents with foresight: predictions of action consequences before execution. However, unreliable foresight can be ignored, misused, or even degrade downstream decision-making. In this paper, we introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen. WorldEvolver integrates three modules: (i) Episodic Memory, which exploits real action transitions through retrieval-based simulation; (ii) Semantic Memory, which extracts persistent heuristic rules from prediction-observation mismatches; and (iii) Selective Foresight, which filters low-confidence predictions before integrating them into agent reasoning context. We evaluate WorldEvolver on ALFWorld and ScienceWorld, measuring world model prediction accuracy on Word2World and downstream agent success rate on AgentBoard. Extensive experiments show that WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate, demonstrating that test-time memory revision enhances both predictive fidelity and planning performance.

Problem

• world models
• LLM agents
• long-horizon planning
• prediction reliability
• frozen parameters

Innovation

• self-evolving world model
• episodic memory
• semantic memory
• selective foresight
• LLM agent planning