Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modeling

2026-07-05

AI Summary

Existing world models often rely on high-dimensional visual representations that contain substantial control-irrelevant information, which hinders decision-making efficiency and task generalization. This work proposes a closed-loop co-adaptation mechanism between the agent and the world model, leveraging adaptive curriculum-guided active exploration, structured observation modeling, and distillation of task-sufficient representations to learn minimal and compact latent variables for the world model. The approach significantly improves sample efficiency on continuous control and robotic manipulation benchmarks and demonstrates strong generalization to novel skills, unseen object-skill combinations, and out-of-distribution tasks.

Abstract

Learning and planning in imagination using world models provides an effective paradigm for training agents for decision-making. However, existing approaches often rely on high-dimensional latent spaces or generic visual embeddings that retain many factors irrelevant to control, limiting efficiency and generalization across tasks. To this end, we study how agents can learn world models with representations that are task-specific, minimal, and sufficient for decision-making. We achieve this via a closed-loop synergy between the agent and the world model, in which structured world-model learning distills task-sufficient representations from informative interaction data. On the agent side, agents actively probe the environment to collect informative trajectories that expose task-relevant latent factors, guided by an adaptive curriculum. On the world-model side, we learn structured representations over observations to distill compact, task-sufficient latent states from the collected interaction data. This synergy enables the empirical recovery of task-sufficient latent representations that capture all control-relevant factors. Leveraging these representations, the resulting policies achieve improved sample efficiency and generalization, including generalization across skills, object-skill compositions, and previously unseen tasks on standard continuous-control and robotic-manipulation benchmarks.

Problem

• world models
• task-sufficient representations
• sample efficiency
• generalization
• latent representations

Innovation

• task-sufficient world models
• agentic exploration
• structured modeling
• latent representation learning
• sample-efficient reinforcement learning