FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models

2026-07-16

AI Summary

This work addresses the limitations of existing vision-language-action models, which lack forward-looking predictive capabilities for environmental dynamics, and current visual foresight methods, which, while capable of predicting future states, offer no explicit guidance on motion trajectories. To bridge this gap, we propose the first unified framework that jointly learns future feature prediction and sparse 2D point trajectory tracking, incorporating explicit spatiotemporal supervision to enhance planning in continuous action policies. Our approach introduces compact foresight tokens and a lightweight future-conditioned cross-attention mechanism, enabling coherent reasoning between target states and motion paths. Evaluated on the LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus benchmarks, the method achieves state-of-the-art performance and demonstrates strong zero-shot generalization capabilities.

Abstract

Vision-Language-Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio-temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact geometric motion, and couples both through a lightweight future-conditioned cross-attention module that enables consistent reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus demonstrate state-of-the-art performance and strong zero-shot generalization. Project page is available at https://liauto-research.github.io/FoMoVLA.

Problem

• Vision-Language-Action models
• visual foresight
• motion guidance
• visuomotor policy
• future prediction

Innovation

• visual foresight
• motion guidance
• vision-language-action models
• sparse point tracking
• future-conditioned cross-attention