VectorWorld: Efficient Streaming World Model via Diffusion Flow on Vector Graphs

2026-03-18

AI Summary

This work addresses key challenges in generative world models for closed-loop autonomous driving evaluation, including history-agnostic initialization, high latency in multi-step sampling, and kinematically infeasible long-horizon trajectories. The authors propose VectorWorld, a streaming world model that enables real-time, long-horizon closed-loop simulation through vector-graph diffusion flows, incrementally generating ego-centric 64m×64m lane-agent map tiles. Core innovations include a motion-aware gated VAE with history-conditioned initialization, a single-step masked completion mechanism driven by an edge-gated relational DiT, and a physics-aligned ΔSim non-ego policy coupled with differentiable kinematic logic shaping. Evaluated on Waymo Open Motion and nuPlan, the method significantly improves map structural fidelity and initialization validity, enabling stable, real-time closed-loop rollouts exceeding one kilometer.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Closed-loop evaluation of autonomous-driving policies requires interactive simulation beyond log replay. However, existing generative world models often degrade in closed loop due to (i) history-free initialization that mismatches policy inputs, (ii) multi-step sampling latency that violates real-time budgets, and (iii) compounding kinematic infeasibility over long horizons. We propose VectorWorld, a streaming world model that incrementally generates ego-centric $64 \mathrm{m}\times 64\mathrm{m}$ lane--agent vector-graph tiles during rollout. VectorWorld aligns initialization with history-conditioned policies by producing a policy-compatible interaction state via a motion-aware gated VAE. It enables real-time outpainting via solver-free one-step masked completion with an edge-gated relational DiT trained with interval-conditioned MeanFlow and JVP-based large-step supervision. To stabilize long-horizon rollouts, we introduce $Δ$Sim, a physics-aligned non-ego (NPC) policy with hybrid discrete--continuous actions and differentiable kinematic logit shaping. On Waymo open motion and nuPlan, VectorWorld improves map-structure fidelity and initialization validity, and supports stable, real-time $1\mathrm{km}+$ closed-loop rollouts (\href{https://github.com/jiangchaokang/VectorWorld}{code}).

Problem

• closed-loop simulation
• generative world models
• real-time latency
• kinematic feasibility
• history-conditioned initialization

Innovation

• VectorWorld
• streaming world model
• masked completion
• physics-aligned NPC policy
• real-time closed-loop simulation