Problem
Research questions and friction points this paper is trying to address.
world modeling
next-observation prediction
sequential decision making
agent policy
decision-oriented dynamics
Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making
🤖 AI Summary
This work proposes Agent-Authored World Modeling (AAWM), a novel approach that addresses the misalignment between conventional world models—typically trained to predict future observations—and the dynamic representations required for effective agent decision-making. Instead of passively forecasting observations, AAWM enables the agent to actively retrieve relevant transitions from its experience trajectories and synthesize decision-oriented dynamics tailored to its policy needs. By leveraging a large language model to generate supervision signals aligned with strategic objectives, AAWM reframes world model training as an active, policy-driven process. Empirical results across diverse environments and training settings demonstrate that AAWM significantly outperforms traditional methods, confirming that its learning targets more effectively support policy optimization.
📝 Abstract
Recent studies on world modeling for Large Language Model (LLM) agents typically formulate the learning objective as next-observation prediction. However, this objective ties supervision to what a transition happens to reveal, which may omit the dynamics most relevant to the agent's current decision. To bridge this gap, we propose Agent-Authored World Modeling (AAWM), a training procedure that constructs supervision from the policy's own decision needs. Specifically, at each state, the agent identifies what it needs to understand about the environment before acting. These needs drive the retrieval of relevant transition evidence across trajectories, which is then synthesized into training targets that capture decision-oriented dynamics instead of reconstructing the next observation. This aligns the training objective with the dynamics the policy needs before acting, not with the contents of the next observation. Experimental results validate the effectiveness of AAWM across multiple environments and training settings. These results show that decision-aware world-model targets provide a more effective learning signal than next-observation prediction.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Agent-Authored World Modeling
decision-aware dynamics
next-observation prediction
world modeling
LLM agents
🔎 Similar Papers
No similar papers found.
