PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

2026-06-16

AI Summary

This work addresses the challenge of simultaneously achieving low-latency control and high-quality planning in vision-language-action (VLA) models. It proposes a novel paradigm that progressively refines action plans within the latent space of a vision-language model by decoupling visual anchoring from an iterative planning branch. Guided by a lightweight, frozen world model, the approach leverages future observation latents as feedback to iteratively optimize semantic action plans, which are then decoded in parallel to enable low-latency execution. Integrated with process-based reward reinforcement learning using causally refined grouping, the method achieves state-of-the-art performance on the LIBERO benchmark, significantly outperforming existing approaches while maintaining computational efficiency and long-horizon reasoning capabilities.

Abstract

Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation. Directly decoding actions from vision-language backbone representations enables low-latency control, whereas explicit reasoning through textual chains, pixel-level subgoals, or action search can improve planning but incurs substantial latency and computational cost. We propose PearlVLA, a VLA framework that moves deliberation into the latent space of a vision-language model (VLM). PearlVLA separates VLM meta-query representations into a fixed visual grounding branch and an iterative latent plan branch. At each refinement round, a plan-conditioned world query probes a lightweight frozen latent world model for an action-free future observation latent, which is fed back to guide plan refinement. A future-guided RefineNet then applies scheduled residual updates to progressively refine a coarse semantic draft into a fine-grained latent action plan. The refined plan after K rounds is then decoded in parallel into an action chunk for low-latency execution. We further introduce Causal Refinement-Grouped Process-Reward RL to optimize the latent refinement process with rewards from longer-horizon imagined futures induced by latent plan edits. Empirical evaluations on the LIBERO benchmark demonstrate that PearlVLA achieves state-of-the-art performance among existing methods.

Problem

• Vision-Language-Action
• action planning
• latency
• deliberation
• embodied AI

Innovation

• latent space planning
• vision-language-action models
• iterative refinement
• future-guided RefineNet
• process-reward reinforcement learning