SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds

2026-04-09

AI Summary

This work addresses the challenges of data scarcity and sim-to-real gaps in deformable object manipulation, which arise from dynamic changes in shape, contact, and topology. The authors propose a physics-aligned, zero-shot data augmentation paradigm that establishes a real-simulation-real closed-loop data engine. Leveraging only a few real-world demonstrations, the method constructs a metric-consistent digital twin to calibrate an elastic dynamics model and employs a diffusion model to generate high-quality, physically plausible trajectories. Remarkably, policies trained exclusively on this synthetic data achieve a 90% zero-shot success rate in the real world, equivalent to using 15 times more real demonstration data, and yield a 50% improvement in generalization performance.

Abstract

Robotic manipulation with deformable objects represents a data-intensive regime in embodied learning, where shape, contact, and topology co-evolve in ways that far exceed the variability of rigids. Although simulation promises relief from the cost of real-world data acquisition, prevailing sim-to-real pipelines remain rooted in rigid-body abstractions, producing mismatched geometry, fragile soft dynamics, and motion primitives poorly suited for cloth interaction. We posit that simulation fails not for being synthetic, but for being ungrounded. To address this, we introduce SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world. Given limited demonstrations, the system digitizes scenes into metric-consistent twins, calibrates deformable dynamics through elastic modeling, and expands behaviors via diffusion-based trajectory generation with quality filtering. This pipeline transforms sparse observations into scaled synthetic supervision with near-demonstration fidelity. Experiments show that policies trained on purely synthetic data achieve parity with real-data baselines at a 1:15 equivalence ratio, while delivering 90% zero-shot success and 50% generalization gains in real-world deployment. These results validate physics-aligned simulation as scalable supervision for deformable manipulation and a practical pathway for data-efficient policy learning.

Problem

• deformable objects
• sim-to-real
• physics simulation
• robotic manipulation
• data efficiency

Innovation

• physics-aligned simulation
• deformable object manipulation
• zero-shot data scaling
• diffusion-based trajectory generation
• real-to-sim-to-real transfer