Problem
Research questions and friction points this paper is trying to address.
world model
physical interaction
planning-oriented
task-consistent
entity role
Beyond Instance Slots: Semantically Rich World Models for Physical Interaction Planning
🤖 AI Summary
本文提出Semantically Rich World Model，通过定义五种功能角色来解决物理交互规划中任务一致性的问题，改进了实体在任务上下文中的作用表达。
📝 Abstract
World models for physical interaction are typically trained to predict future observations or latent features; however, a planning-oriented model must answer a fundamentally different question: whether a candidate action produces a task-consistent future while preserving essential relations.Monolithic state representations obscure the underlying entities, while standard instance-level object slots merely identify \emph{what} is present without specifying \emph{what role} each entity plays in the task context. To bridge this gap, we present the Semantically Rich World Model (SR-WM), a task-conditioned world model structured around five functional roles: gripper, target, goal, relation, and phase.Within SR-WM, a visual entity encoder extracts soft entity hypotheses from pretrained patch features, allowing segmentation masks to serve as optional proposal priors without mandating them as required state representations or inference inputs.A role binder subsequently maps these hypotheses to task-specific roles, while an action-conditioned dynamics model predicts role transitions alongside fine-grained semantics, including grasp/contact, predicate establishment, relation preservation, fixture state, and phase change.Crucially, this unified role state grounds downstream multi-candidate action generation, stage-aware reranking, and violation-aware suffix resampling.Our comprehensive evaluation protocol spans all four LIBERO simulation suites, cross-suite transfer, perception diagnostics, and action-sensitivity analysis.Ultimately, this formulation transforms object-centric prediction into a semantic interface linking visual dynamics with planning-oriented decision making.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Semantically Rich World Model
functional roles
visual entity encoder
role binder
action-conditioned dynamics model
🔎 Similar Papers
No similar papers found.
