Towards Predictive, Aligned, and Scalable Robot Learning

2026-07-13

AI Summary

This work addresses the challenge that existing robot learning methods struggle to simultaneously achieve predictive capability, cross-modal alignment, and scalability in complex physical interactions and long-horizon tasks. The authors propose Lumo-2, a latent-space world-action model that structures the latent representation to jointly model world dynamics and generate actions. Central to this approach is a novel hypothesis that the geometric structure of the latent space fundamentally determines action quality. Lumo-2 further incorporates a multi-stage modality pre-alignment mechanism to ensure consistency across vision, language, and action modalities, and introduces action tokenization with lightweight dynamic inference for efficient predictive control. Experiments demonstrate that Lumo-2 significantly outperforms current vision-language-action (VLA) and world-action model (WAM) approaches on real-world dexterous manipulation and long-horizon planning tasks, underscoring the critical role of structured alignment and predictive reasoning in embodied intelligence.

Abstract

Learning, at its core, extends beyond memorization to the ability to reason and solve novel problems by navigating a space of possibilities. We introduce Lumo-2, a latent world-action model that generates actions by reasoning over world dynamics in latent space. The learned latent world dynamics capture physically grounded visual transitions, naturally encoding future possibilities and providing a unified substrate for cross-modal alignment. This formulation enables predictive reasoning akin to world modelling while remaining lightweight and focused on physical dynamics relevant to control. Central to our approach is the hypothesis that action generation quality is governed by the geometry of the latent space. We observe that standard reconstruction-based action tokenization objectives induce representations biased toward low-level signal fidelity, leading to misalignment between reconstruction quality and downstream control performance. To address this limitation, we propose a multi-stage modality pre-alignment strategy in which action representations are progressively aligned with latent world dynamics, vision, and language. This process enforces cross-modal consistency, promotes abstraction, and induces a structured latent space for predictive reasoning. We provide a systematic empirical study of latent world modelling and modality alignment, analyzing their roles in scaling laws and out-of-distribution generalization. Results show that Lumo-2 consistently outperforms strong vision-language-action (VLA) and world-action model (WAM) baselines, with gains on challenging real-world tasks requiring temporal reasoning, physical understanding, or high control complexity, including long-horizon and dexterous manipulation. These findings suggest that structured multimodal alignment and predictive reasoning are fundamental principles for advancing embodied intelligence.

Problem

• robot learning
• action generation
• latent space
• modality alignment
• predictive reasoning

Innovation

• latent world-action model
• predictive reasoning
• modality alignment
• structured latent space
• embodied intelligence