EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields

2026-05-07

AI Summary

Existing robotic world models struggle to preserve precise spatial geometry and fine-grained robot-object interaction dynamics in action-conditioned video generation. To address this, this work proposes an event-aware generative world model that projects actions and kinematic states into structured visual action fields and introduces an event-aware bidirectional fusion module to modulate cross-branch attention, thereby closing the loop between motion control and visual perception. Built upon a pretrained video diffusion model, the approach incorporates structured action field representations, bidirectional attention fusion blocks, and an end-to-end joint optimization framework. Evaluated on the WorldArena benchmark, the method significantly outperforms existing approaches, generating more realistic robot geometries and interaction dynamics.

Abstract

Pretrained video diffusion models provide powerful spatiotemporal generative priors, making them a natural foundation for robotic world models. While recent world-action models jointly optimize future videos and actions, they predominantly treat video generation as an auxiliary representation for policy learning. Consequently, they insufficiently explore the inverse problem: leveraging action signals to guide video synthesis, thereby often failing to preserve precise robot spatial geometry and fine-grained robot-object interaction dynamics in the generated rollouts. To bridge this gap, we present EA-WM, an Event-Aware Generative World Model that effectively closes the loop between kinematic control and visual perception. Rather than injecting joint or end-effector actions as abstract, low-dimensional tokens, EA-WM projects actions and kinematic states directly into the target camera view as Structured Kinematic-to-Visual Action Fields. To fully exploit this geometrically grounded representation, we introduce event-aware bidirectional fusion blocks that modulate cross-branch attention, capturing object state changes and interaction dynamics. Evaluated on the comprehensive WorldArena benchmark, EA-WM achieves state-of-the-art performance, outperforming existing baselines by a significant margin.

Problem

• world model
• video generation
• action-conditioned synthesis
• robot-object interaction
• spatial geometry

Innovation

• Structured Kinematic-to-Visual Action Fields
• Event-Aware Fusion
• Generative World Model
• Video Diffusion
• Robot-Object Interaction