RoboAug: One Annotation to Hundreds of Scenes via Region-Contrastive Data Augmentation for Robotic Manipulation

2026-02-15

AI Summary

This work proposes RoboAug, a novel approach to enhance robotic generalization in unseen environments while reducing reliance on large-scale datasets and perfect object detection. Leveraging only a single bounding box annotation per image, RoboAug employs a pretrained generative model for semantic data augmentation and introduces a plug-and-play region-wise contrastive loss to guide the policy toward task-relevant regions. This method enables the generation of hundreds of diverse training scenes from a single annotated example—a first in the field. Evaluated on three robotic platforms (UR-5e, AgileX, and Tien Kung 2.0), RoboAug significantly improves task success rates in unseen scenarios from 0.09/0.16/0.19 to 0.47/0.60/0.67, outperforming existing data augmentation techniques.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Enhancing the generalization capability of robotic learning to enable robots to operate effectively in diverse, unseen scenes is a fundamental and challenging problem. Existing approaches often depend on pretraining with large-scale data collection, which is labor-intensive and time-consuming, or on semantic data augmentation techniques that necessitate an impractical assumption of flawless upstream object detection in real-world scenarios. In this work, we propose RoboAug, a novel generative data augmentation framework that significantly minimizes the reliance on large-scale pretraining and the perfect visual recognition assumption by requiring only the bounding box annotation of a single image during training. Leveraging this minimal information, RoboAug employs pre-trained generative models for precise semantic data augmentation and integrates a plug-and-play region-contrastive loss to help models focus on task-relevant regions, thereby improving generalization and boosting task success rates. We conduct extensive real-world experiments on three robots, namely UR-5e, AgileX, and Tien Kung 2.0, spanning over 35k rollouts. Empirical results demonstrate that RoboAug significantly outperforms state-of-the-art data augmentation baselines. Specifically, when evaluating generalization capabilities in unseen scenes featuring diverse combinations of backgrounds, distractors, and lighting conditions, our method achieves substantial gains over the baseline without augmentation. The success rates increase from 0.09 to 0.47 on UR-5e, from 0.16 to 0.60 on AgileX, and from 0.19 to 0.67 on Tien Kung 2.0. These results highlight the superior generalization and effectiveness of RoboAug in real-world manipulation tasks. Our project is available at https://x-roboaug.github.io/.

Problem

• robotic manipulation
• generalization
• data augmentation
• object detection
• unseen scenes

Innovation

• RoboAug
• region-contrastive loss
• semantic data augmentation
• minimal annotation
• robotic manipulation