Bridge-WA: Predicting Where and How the World Changes for Robotic Action

2026-07-02

AI Summary

Existing general-purpose vision-language-action models struggle to efficiently model scene changes induced by actions, often relying on computationally expensive generative world models or dense future-frame prediction, which are susceptible to irrelevant visual distractions. This work proposes Bridge-WA, a lightweight framework that, for the first time, decouples future scene changes into “where it changes” (change maps) and “how it changes” (motion flow maps), and introduces compact future tokens as a prior. By distilling knowledge from a frozen teacher model and leveraging the WorldBridge module’s multi-source attention with spatiotemporal bias mechanisms, Bridge-WA guides action decisions without generating full future images. The approach significantly enhances robustness to background variations, lighting changes, and distractors, achieving superior task success rates, progress metrics, and out-of-distribution generalization across VLABench, RoboTwin2.0, LIBERO-Plus, and real-world robotic platforms—particularly excelling under visual distribution shifts.

Abstract

General-purpose vision-language-action models benefit from large vision-language priors, but effective manipulation also requires anticipating action-relevant scene changes. Existing world-action models often rely on large generative world models or dense future rollouts, which are expensive and spend capacity on visual details weakly coupled to control. We present Bridge-WA, a lightweight world-action framework that distills a frozen future-change teacher into three compact priors: future tokens for intended outcomes, change maps for intervention support, and motion-flow maps for local transition direction. A WorldBridge conditions the action transformer on these priors through multi-source attention memories and spatial-temporal biases, while the teacher model is removed at inference. Across VLABench, RoboTwin2.0, LIBERO-Plus and real-robot evaluations, Bridge-WA improves task success, progress, and robustness, with particularly clear gains under out-of-distribution visual shifts. By focusing action generation on where and how the scene will change, Bridge-WA suppresses nuisance appearance factors such as background, lighting, and distractors, leading to better generalization without deployment-time dense future-image generation. Code and visualizations are available at: https://hcplab-sysu.github.io/BRIDGE-WA .

Problem

• world-action models
• scene change prediction
• robotic manipulation
• visual generalization
• action-relevant dynamics

Innovation

• world-action modeling
• knowledge distillation
• change prediction
• vision-language-action
• lightweight inference