FUTURE-VLA: Forecasting Unified Trajectories Under Real-time Execution

2026-02-05

arXiv.org

AI Summary

本文提出PACT-WAM模型，通过紧凑的时间编码方法减少处理成本，同时生成动作轨迹和视觉预测，提高机器人操作的成功率。

Abstract

General vision-language models increasingly support unified spatiotemporal reasoning over long video streams, yet deploying such capabilities on robots remains constrained by the prohibitive latency of processing long-horizon histories and generating high-dimensional future predictions. To bridge this gap, we present FUTURE-VLA, a unified architecture that reformulates long-horizon control and future forecasting as a monolithic sequence-generation task. Adopting a dual-sided efficiency paradigm, FUTURE-VLA leverages a temporally adaptive compression strategy to maximize spatiotemporal information density, enabling the ingestion of extensive multi-view histories while maintaining constant inference latency. Simultaneously, it performs latent-space autoregression to align actionable dynamics with reviewable visual look-aheads in a single forward pass. These real-time predictive capabilities further enable a prediction-guided Human-In-the-Loop mechanism via interactive execution gating, allowing operators to dynamically validate behaviors based on interpretable future previews. Extensive evaluations demonstrate that FUTURE-VLA establishes new state-of-the-art performance, attaining success rates of 99.2% on LIBERO, 75.4% on RoboTwin, and 78.0% on a real-world Piper platform, all with a $16\times$ extended spatiotemporal window while maintaining the inference latency of a single-frame baseline.

Problem

• Robot Manipulation
• Temporal Context
• Visual Foresight
• Processing Costs

Innovation

• PACT-WAM
• Hierarchical History Encoding
• Conditional Flow Sampling
• Shared Flow Module
• Proposal Review