RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generation

2026-02-18

AI Summary

This work addresses the scarcity of diverse, physically feasible interaction data that limits real-world robotic manipulation. To this end, we propose RoboGene, an agent-based automated task generation framework that uniquely integrates diversity-driven sampling with self-reflective physical constraint validation, augmented by a human-in-the-loop iterative refinement mechanism. RoboGene efficiently generates high-quality manipulation tasks for single-arm, dual-arm, and mobile robots, enabling large-scale collection of real-world trajectories (18k in total). The resulting dataset significantly outperforms those generated by state-of-the-art models such as GPT-4o and Gemini 2.5 Pro. Vision–language–action (VLA) models pretrained on this data demonstrate markedly higher success rates and superior generalization capabilities on real-world manipulation tasks.

Technology Category

• Intelligent Robots: Manipulation
• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike data collection from web in vision or language, robotic data collection is an active process incurring prohibitive physical costs. Consequently, automated task curation to maximize data value remains a critical yet under-explored challenge. Existing manual methods are unscalable and biased toward common tasks, while off-the-shelf foundation models often hallucinate physically infeasible instructions. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks across single-arm, dual-arm, and mobile robots. RoboGene integrates three core components: diversity-driven sampling for broad task coverage, self-reflection mechanisms to enforce physical constraints, and human-in-the-loop refinement for continuous improvement. We conduct extensive quantitative analysis and large-scale real-world experiments, collecting datasets of 18k trajectories and introducing novel metrics to assess task quality, feasibility, and diversity. Results demonstrate that RoboGene significantly outperforms state-of-the-art foundation models (e.g., GPT-4o, Gemini 2.5 Pro). Furthermore, real-world experiments show that VLA models pre-trained with RoboGene achieve higher success rates and superior generalization, underscoring the importance of high-quality task generation. Our project is available at https://robogene-boost-vla.github.io.

Problem

• robotic manipulation
• task generation
• data scarcity
• physical feasibility
• diversity

Innovation

• diversity-driven sampling
• self-reflection mechanism
• human-in-the-loop refinement
• physically plausible task generation
• VLA pre-training