Right-Side-Out: Learning Zero-Shot Sim-to-Real Garment Reversal

2025-09-19

AI Summary

This paper addresses the challenging robotic manipulation task of garment flipping—characterized by high dynamics, severe occlusion, and frequent contact changes—by proposing a simulation-to-reality zero-shot transfer framework that requires no human demonstrations. Methodologically: (1) A task-structure-driven two-stage decoupling strategy decomposes garment flipping into key subtasks—dragging, folding, and insertion-pulling; (2) A bimanual robot primitive parameterized by keypoints drastically reduces action-space dimensionality; (3) A high-fidelity thin-shell deformation simulator, built upon GPU-accelerated Material Point Method (MPM), is integrated with depth-based inference, keypoint annotation, and binary mask generation to enable end-to-end policy learning. The policy trained exclusively in simulation achieves 81.3% success rate when deployed zero-shot on a real dual-arm robot platform, robustly handling severe visual occlusion and dynamic contact transitions.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Turning garments right-side out is a challenging manipulation task: it is highly dynamic, entails rapid contact changes, and is subject to severe visual occlusion. We introduce Right-Side-Out, a zero-shot sim-to-real framework that effectively solves this challenge by exploiting task structures. We decompose the task into Drag/Fling to create and stabilize an access opening, followed by Insert&Pull to invert the garment. Each step uses a depth-inferred, keypoint-parameterized bimanual primitive that sharply reduces the action space while preserving robustness. Efficient data generation is enabled by our custom-built, high-fidelity, GPU-parallel Material Point Method (MPM) simulator that models thin-shell deformation and provides robust and efficient contact handling for batched rollouts. Built on the simulator, our fully automated pipeline scales data generation by randomizing garment geometry, material parameters, and viewpoints, producing depth, masks, and per-primitive keypoint labels without any human annotations. With a single depth camera, policies trained entirely in simulation deploy zero-shot on real hardware, achieving up to 81.3% success rate. By employing task decomposition and high fidelity simulation, our framework enables tackling highly dynamic, severely occluded tasks without laborious human demonstrations.

Problem

• Solving zero-shot sim-to-real garment reversal task
• Decomposing dynamic manipulation into structured bimanual primitives
• Overcoming visual occlusion through depth-inferred keypoint parameterization

Innovation

• Task decomposition into bimanual primitives
• Custom GPU-parallel MPM simulator
• Zero-shot sim-to-real depth policy