Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation

2026-06-29

AI Summary

This work addresses the limitations of existing world model–based visual navigation approaches, which typically decouple goal intent verification from trajectory generation, leading to computational redundancy and inconsistencies between actions and visual predictions. To overcome these issues, the authors propose SWAM—a task-driven, joint observation–action generation framework that, for the first time, enables end-to-end zero-shot cross-environment generalization from only monocular RGB inputs. SWAM simultaneously synthesizes intermediate RGB-D sequences and corresponding action trajectories in a single forward pass, starting solely from source and goal RGB images. The method incorporates a vision-guided action refinement module and a trajectory-scale regularization loss, while leveraging depth pseudo-labels to internalize spatial priors. Experimental results demonstrate that SWAM significantly outperforms current two-stage planners in terms of success rate, trajectory accuracy, and inference efficiency.

Abstract

Existing world model-based planners for visual navigation typically follow a verification-centric paradigm, decoupling goal intent from trajectory synthesis. This approach suffers from candidate dependence, heavy computational overhead, and inconsistencies between sampled actions and predicted visuals. To address these issues, we propose SWAM (Spatial-perceiving World Action Model), a task-centric joint observation-action generation framework. Given start and goal RGB observations, SWAM performs single-pass inference to simultaneously generate intermediate RGB-D sequences and corresponding action trajectories, promoting goal-consistent trajectory generation and improved spatial feasibility. While SWAM leverages depth pseudo-labels during training to internalize spatial priors, it requires only monocular RGB input at inference time. We further introduce a visual-guided action refinement module and a trajectory-scale regularization loss to enforce fine-grained alignment between motion and visual cues while stabilizing predictions across varying distances. Extensive experiments show that SWAM significantly outperforms state-of-the-art two-stage planners in success rate, trajectory accuracy, and inference efficiency, while demonstrating robust zero-shot generalization to unseen environments.

Problem

• visual navigation
• world model
• trajectory generation
• action-visual consistency
• embodied AI

Innovation

• world model
• embodied navigation
• joint observation-action generation
• spatial perception
• zero-shot generalization