Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models

2026-06-16

AI Summary

This work addresses the challenges posed by the high heterogeneity, expensive acquisition, and limited diversity of robotic manipulation data, which hinder the generalization of foundation models. Building upon Qwen-VL, the authors propose a unified vision-language-action foundation model featuring a three-dimensional alignment framework across representation, motion, and behavior. By integrating a synthetic pipeline that translates human hand videos into robot trajectories and a robust heterogeneous data cleaning process, the model achieves large-scale open-domain pretraining—spanning tens of thousands of hours—without requiring proprietary datasets. It significantly outperforms existing methods such as π0.5 on out-of-distribution benchmarks like RoboCasa365 and LIBERO-Plus, ranks first on RoboChallenge with a 20% performance gain, and demonstrates emergent capabilities—including zero-shot instruction following, perturbation robustness, error recovery, and cross-platform transfer—on real-world platforms such as AgileX ALOHA, Franka, UR, and ARX.

Abstract

Foundation models in language and multimodality achieve strong generalization by aligning heterogeneous data under a unified formulation and training at scale. In this report, we investigate whether this scaling recipe can be applied to robotic manipulation to achieve genuine generalization. This is challenging because, unlike text, manipulation data is heterogeneous by nature, expensive to collect, and narrow in diversity, making alignment and scale simultaneously difficult. We present Qwen-RobotManip, a generalizable Vision-Language-Action foundation model built on Qwen-VL. Qwen-RobotManip introduces a unified alignment framework across the representation, motion, and behavioral dimensions of manipulation, making large-scale multi-source training coherent rather than conflicting. This alignment capability in turn enables Qwen-RobotManip to absorb manipulation data at a scale that prior training regimes could not sustain. A human-to-robot synthesis pipeline converts egocentric hand demonstrations into robot trajectories across 15 platforms, and a rigorous curation pipeline harmonizes heterogeneous datasets. Using only open-source datasets and human videos without proprietary data collection, Qwen-RobotManip constructs a ~38,100-hour pretraining corpus and exhibits emergent generalization capabilities, including zero-shot instruction following, robustness to perturbations, reactive error recovery, and cross-embodiment transfer. We find that standard benchmarks fail to capture pretraining quality and instead adopt OOD settings including RoboCasa365, LIBERO-Plus, EBench, RoboTwin-Clean2Rand, RoboTwin-IF, and RoboTwin-XE. Qwen-RobotManip substantially outperforms prior state-of-the-art models, including $π$0.5, across all OOD settings, ranks 1st in RoboChallenge with a 20% relative improvement, and is validated on real-robot platforms including AgileX ALOHA, Franka, UR, and ARX.

Problem

• robotic manipulation
• foundation models
• data alignment
• scale
• generalization

Innovation

• alignment
• foundation model
• robotic manipulation
• zero-shot generalization
• cross-embodiment transfer