MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation

2026-03-17

AI Summary

This work addresses the common reliance of simulation-to-reality (sim-to-real) transfer on real-world data or task-specific fine-tuning by proposing a purely simulation-driven zero-shot transfer approach. Leveraging procedurally generated expert trajectories—comprising 1.8 million highly diverse demonstrations—and integrating the Molmo2 vision-language model with a flow-matching action head and variants of π₀/SPOC policies, the method achieves cross-domain manipulation without any real-world fine-tuning. Evaluated on Franka FR3 and RB-Y1 platforms, it attains a 79.2% success rate on previously unseen objects and environments, demonstrating for the first time that large-scale, diverse simulation alone can effectively enable zero-shot transfer to real-world static and dynamic manipulation tasks.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

A prevailing view in robot learning is that simulation alone is not enough; effective sim-to-real transfer is widely believed to require at least some real-world data collection or task-specific fine-tuning to bridge the gap between simulated and physical environments. We challenge that assumption. With sufficiently large-scale and diverse simulated synthetic training data, we show that zero-shot transfer to the real world is not only possible, but effective for both static and mobile manipulation. We introduce MolmoBot-Engine, a fully open-source pipeline for procedural data generation across robots, tasks, and diverse simulated environments in MolmoSpaces. With it, we release MolmoBot-Data, a dataset of 1.8 million expert trajectories for articulated object manipulation and pick-and-place tasks. We train three policy classes: MolmoBot, a Molmo2-based multi-frame vision-language model with a flow-matching action head; MolmoBot-Pi0, which replicates the $π_0$ architecture to enable direct comparison; and MolmoBot-SPOC, a lightweight policy suitable for edge deployment and amenable to RL fine-tuning. We evaluate on two robotic platforms: the Franka FR3 for tabletop manipulation tasks and the Rainbow Robotics RB-Y1 mobile manipulator for door opening, drawer manipulation, cabinet interaction, and mobile pick-and-place. Without any real-world fine-tuning, our policies achieve zero-shot transfer to unseen objects and environments. On tabletop pick-and-place, MolmoBot achieves a success rate of 79.2% in real world evaluations across 4 settings, outperforming $π_{0.5}$ at 39.2%. Our results demonstrate that procedural environment generation combined with diverse articulated assets can produce robust manipulation policies that generalize broadly to the real world. Technical Blog: https://allenai.org/blog/molmobot-robot-manipulation

Problem

• sim-to-real transfer
• zero-shot manipulation
• robot learning
• simulation
• generalization

Innovation

• zero-shot transfer
• large-scale simulation
• procedural data generation
• vision-language policy
• sim-to-real