ShapeGen: Robotic Data Generation for Category-Level Manipulation

2026-04-16

AI Summary

This work addresses the challenge of poor generalization in robotic manipulation strategies across objects of the same category but with significant geometric variations in real-world settings. To overcome this, the authors propose ShapeGen, a simulation-free, category-level 3D manipulation data generation framework that operates in two stages: constructing a functional-aware shape library and generating manipulation demonstrations. By modeling spatial deformations between functionally corresponding points, ShapeGen builds a plug-and-play, physically plausible 3D shape repository, enabling the synthesis of diverse and effective manipulation trajectories from only a few human annotations. Real-world experiments demonstrate that policies trained with ShapeGen exhibit substantially improved generalization across geometrically diverse instances within the same object category.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Manipulation policies deployed in uncontrolled real-world scenarios are faced with great in-category geometric diversity of everyday objects. In order to function robustly under such variations, policies need to work in a category-level manner, i.e. knowing how to interact with any object in a certain category, instead of only a specific one seen during training. This in-category generalizability is usually nurtured with shape-diversified training data; however, manually collecting such a corpus of data is infeasible due to the requirement of intense human labor and large collections of divergent objects at hand. In this paper, we propose ShapeGen, a data generation method that aims at generating shape-variated manipulation data in a simulator-free and 3D manner. ShapeGen decomposes the process into two stages: Shape Library curation and Function-Aware Generation. In the first stage, we train spatial warpings between shapes mapping points to points that correspond functionally, and aggregate 3D models along with the warpings into a plug-and-play Shape Library. In the second stage, we design a pipeline that, leveraging established Libraries, requires only minimal human annotation to generate physically plausible and functionally correct novel demonstrations. Experiments in the real world demonstrate the effectiveness of ShapeGen to boost policies' in-category shape generalizability. Project page: https://wangyr22.github.io/ShapeGen/.

Problem

• category-level manipulation
• in-category generalizability
• shape diversity
• robotic data generation
• 3D object manipulation

Innovation

• ShapeGen
• category-level manipulation
• shape variation
• function-aware generation
• spatial warping